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Abstract 

The goal of this dissertatjdn Is to determlhe the extent to which learning debugging in the 
context of LOGO prdgrammlng improves children's debugging In other programming and non- 
prograrnrnlng contexts. The approach involves detailed task analysis of debugging (in the 
form of a computer slmulatloh model), development of model-based Instructional guidelines 
for teaching children debugging skills they do not learn "by discovery." and assessrnent of 
the debugging skills children are able to transfer to other prdgramrnihg and hdh-prbgramming 
tasks, fwehty-twd 8- to 11 -year-old students took two 25 hour LOGO courses. Half of the 
students were taught debugging In the context of a LOGO graphics course first and ther. a 
LOGO list-prbcesslhg course. The other half were taught debugging In the same two inini- 
courses but in the reverse order: Debugging skills were tested at thfee timee during each 
mini-cburse. The perfbrmahce of chlldreh taking tests In the first rnNiN:ourse was compared 
with the performance of children taking the same tests In the second ' mM-course to reveal 
the transfer frbrn one LOGO dbrnain to the bther. Debugging on nolt-programrnlrig tasks 
was assessed prior to the first mini-course, between mini-courses, and after the last mini- 
course to assess more remote transfer of debugging skills. Assessments of students' 
debugging skill revealed large savings from the first to the second mini-course. Students' 
increasing use of selective search strategies increased the accuracy, efficiency, and speed of 
their debugging. Corresponding Irnproverrients were demonstrated on a variety 6i tasks 
requiring debugging bf hbh-cbrhpUter directibhs. Children shifted from exhaustive to selective 
search strategies which increased the accuracy, effictency. and speed with which they 
debugged written directions. Thus, the debugging strategies learned frbm explicit Instructibh 
in the the first cbrtipUier prbgrammihg mlni-cbUrse Had a positive impact oh children's 
debugging In both new programming and non-programming situations. 
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1. SeeNihg sKiil transfer from eomputer programming 

Transfer of training Is the educator's dream but the researcher's hightrnare. Cieariy, 
teachers will b9 most effective when the learhlng of one Instructlbhal unit contributes to the 
learning of a subsequent unit or when a current lesson is facilitated by emphasizing reievarrt 
previous learning. At the same time, teachers hope that schbbi lessons can be useful for 
sblvlhg ''real world'' problems. Essentially, educators strive to teach transferable skills 
because It wbuid be Irnpossible to teach all the knowledge students wHl need in their 
lifetimes. Despite the educator's dreaiti. transfer is very diffrcuit for researchers to 
demonstrate. Studies of human problem solving consistently find that experience with one 
problem rarely yields transfer to other prbblerns even If they are sirnllar. In the face of 
these negative results, could more distant transfer ever be expected? Or should educatori 
abahden their dream? 

The domain of LOGO computer programming is an interesting case In point because of 
the striking contrast between the grahdibse clairns for transfer of high-level thinking skills 
from programming experience and the largely negative results of LOGO transfer studies. 
This dissertatibn fbcuses bh bhe central aspect of learning to ^ragmmi iearning ^ow fo 
debug faulty programs, this sspect has been identified as one of tfrn-^poii if f u l ideas" that 
can generalize far beyond the prbgrarnrning context in which it is acqtriretf' (F^apert. 19610). 
In more practical terms, debugging Is a good candidate for special fbct» since failure tb 
acquire good debugging skills could represent a "significant bottleneck to the development 
of programrnlrig cbmpetericies arid whatever thlrikirig skills rnay be fostered through high-level 
programming proficiencies." ^ 

Transfer of debugging skill is expected because childreri discover iri prbgrarnrnirig 
environments that 1) complex procedures can be cbristructed from sUbcbrripbrierits. 2) errors 
may derive from just a few buggy components, and 3) sources of error can be detected 
and corrected (debugged). In bther erivirbrirrierits. childreri gerieralize that i) rnbst of what 
they do is constructed from smaller cbrhpbhents, 2) errors may derive frorn jUst a few buggy 
components^ and 3) sources of error can be detected and corrected (debugged). Papert 
(1S80) suggesis the pbwer of the debugglrig Idea by sayirig: "Errors beriefit us because 



This comment came from an Insighiful bui andnymdlis reviewer. 
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they iud us to study what happened, to anderstand what went wrbhg^ and, through 
underitahdihg, to fix It. Experience with computer prbgrarnmlng leidi children more 
effectively than any other activity to 'believe in' debagging;'' 

Unfortunately, this notion of transfer and the term "debugging" Itself are ambiguous; this 
malces It difficult to evaluate claims for the cognitive consequences of iearning debugging in 
a LOGO environrnent. Debugging can be interpreted along a wide spectrum, ranging from 
an aii-encdmpassing hbtibh bf self-lrhprovemeht, tb a mbre restricted view of eliminating 
faulty components In physical or mental procedures^, to a constrained definition that is 
closest to its origins In computer prbgrarnrnlhg: It Is what bhe dbes tb get a milfUhctlbhIhg 
(buggy) computer program to work correctly. The potential for transfer of debugging skills Is 
also difficuit to interpret. The goal bf this dissertation Is ib use a detailed perfbrmance 
thebry as the basis for determining the extent to which learning debugging in a LOGO 
context improves chiidren's debugging in other prbgrarnrning and nbn-prbgrarnrning domains. 

In this chapter. I will describe the LOGO computer programming ianguage and the high- 
levei thinking skills educators dream that students acquire and transfer to other domains. 
Then I will briefly review the literature on the reality of transfer from LOOO programming 
experience and propose severai reasons for the primarily negative results bf the LOGO 
transfer studies. Finally. I will develbp ah apprbach for facilitating and assessing transfer in 
the LOGO domain based on a review of several transfer success storl(93 from the adult 
prbbiem-sbivihg literature. 

1.1 The dream 

Proponents of LOGO claim that prbgramrnihg experience can expand children s intellectual 
power (Papert. 1972. 1980). They claim LOGO becomes a tool for life: once it becomes a 
mental model for the chiid, his thinking processes become more conscious and he can 
master cbncepts previously thought tbb abstract. The key element bf LOGO Is said to be 
that powerful ideas are embedded in a simple language. The next two sections will 
describe the powerfui constructs of this simple language and then enurnerate the powerful 
ideas children may be able to learn from experience with this domain. 



- For example, jaggling is one of Paoert's favorite domams for demonstrating learning via debugging. Also 
P^9wr_ & Burton (1978) have convincingly demonstrated the need for debugging in children's acquisition bf 
arithmetic procedures. 
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1.1.1. A almpli languaga with powerful constructi 

At firat glanci, L090 appaara to ba a simplistic prbgrammlhg languaga designed solely to 
enable young ehlldrin with little training to create interesting graphic effects. Yet beneath 
Its facade of slmpjlcltyi LOGO is a powerful programming language which aiiows 
subprocedures, variables, and recurslbn In grajDhlcs as well as llst-prbcessihg. The following 
discussion will demdhstrate the simple yet powerful nature of LOGO by briefly Introducing the 
graphics and list-processing domains; 

In LOGO graphics, the user directs the movement of the cursor around the screen using 
four basic commands: FD (forward), BK (bacl(), LT (left turn), and RT (right turn). Each of 
these commands requires a humerlcai argument to Indicate the distance to nidve (for FD 
and BK) or the number of degrees to turn (for LT and Rt). in addition, PU (penup) and 
PD (pehdowh) cohtrbl the pbsltlbh bf ah Imaginary pen; when the pen Is dbwh, any cursbr 
movement leaves a trace on the screen: With these six, short, semantlcally meaningful 
cbmmands, ybung children can create pleasing designs in LOGONS interactive mode. 

However, children can begin to utiiize the power of LOGO by oslng Its simple screen 
editor tb write and revise programs. These programs can be callwt. interactively or from 
within other programs. Lb(3b also has primitives to direct the flovi cofttfol; these ihciude 
REPEAT h [list of cbmmandsj. which repeats the list of ca imn i m ds n times; iF 
<cohdltiohal> THEN < commands > ELSE < commands >. which Is a basic condltlbhal 
statement: and STOP, which stops the execution of the current program and returns control 
tb the calling prbcedure. 

The example programs in Figure 1 demonstrate the basic graphics capabiiities of LOGO 
The prbcedUre defihitlbhs are listed bh the left. For easier reading, sbrne cbrnrnarids are 
Indented and arranged on separate lines. The interactive calls and bUtcbrties bf the 
procedures are on the right. The starting position of the turtie is indicated by an arrow. In 
Figure 1a, the REPEAT statement is used to write programs tb draw a dlambrid arid a 
curve. The leaf program is written using a repeated curve. These prbgrartis are cbmbihed 
tb mai<e a program to draw a flower: Each of the programs includes a variable (:b. :C. :B. 
and :A) sb that brie cari draw fibwers bf various sizes. The first line of cbmmarids draws 
the ieaves and the stem portions below and between the leaves. The secbrid line draws 
the remainder of the stem, and the third line uses a REPEAT statement to draw ien 
dlartlbrids at equally spaced brieritatibris. Figure ib iiiustrates how a recursive procedure 
can generate a row of flowers of decreasirig size. 
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Insert Figure 1 about here 



There Is more to bd@d than graphics, however. The llst-processlhg capabilities of LQGQ 
are similar to USP, the artificial Intelligence language dh which it Is based. The user can 
PRiNT Ihfdrniatrdh to the computer screen, MAKE globaj variable assignments, and READ 
keyboard Input. OUTPUT rnakes the result of a procedure available to a calling procedure. 
LOGO also has cdmrnahds to separate arid join text; for example. FIRST takes the first 
element of a word or list and BUTfIRST takes the rest. WORD cornbines Its argurnents 
Into a LOGO word and SENTENCE combines its arguments Into a LOGO list. Here again, 
the Cdmrnahds are designed to be semantlcaiiy meaningful. However, the syntax for the list- 
processing commands Is more cdrnpllcated than for the graphics commands because of the 
puhctuatidh necessary to distinguish commands from variables from text. 

The same powerful constructs described above can be used for list-processing. The 
e)(ample programs in Figure 2 dernonstrate the basic list-processing capabiiities of LOGO. 
Here, the procedure definitions are listed above the Interactive calls and output, in Figure 
2a, the PIGGY program translates an input word Into pigiatin by combining (WORD) ail but 
the first letter (BOTFlRST), then the first letter (FiRST). and finally the "ay" ending. 
OUTPUT Is used to pass the resulting word tc another cdrnrnahd. in this case. PRINT. 
Figure 2b lilustrates the use of recursion to translate an entire sentence into pigiatin. 
Finaiiy, Figure 2c shows a prograrn written to rnake the other two programs accessible to a 
halve User, it asks for a sentence to translate, does the translation, and offers an option to 
continue. The user's answer Is set equal to the global variable Y and Is then tested by the 
cohditlbrial In the next line. A "ho" answer yields a farewell staterrient; any other answer 
causes the PIGLATIN program to run again. 

insert Figure 2 about here 



As the graphics and list-processing examples dernonstrate, LOGO offers ah easy 
introductidh to prdgramming as well as opportunities to develop advanced programming skills. 
However, the great expectation is that children will learn thini(ing skills In addition to 
prdgrarnrning skills. Pea & Kurland (1984) cbrrimeht on the widespread belief that 

"...through learning to program, children are learning rriuch rndre than 
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prbgrammingt far mora than programming *f acts'. It Is said that children 
acQQlro po^rerfuity general higher c^^ skills such as planning abllltlw^ problem- 
solving heartetlca, and reflectiveness on the revlslonary chjy'acter of the ^robleni 
soMng pmcew Itself. This beiief, ajthough now In jts appljcatlon to this doma[n^ Is 
ah old Idea In a new costume v^hich has been worn often before. J n Its common 
extreme forfrii It Is based on an assumption about learning - i^^t jspontaneous 
experience with l powerful symbolic system will have beneficial cognitive 
consequences, especially for higher order cognitive skills. ^Irnilar arguments have 
been offered In centuries past for rnatherhatics, logic, writing systems, And Latin 
(e.g., see Brurier, 1966; Gble & Griffin, 1980; Goody, 1977; Olson, 1976; Ong, 
1982; Vygotsky, 1978)." 

1.1.2. The pdsslbllity of powerful ideas 

The ''powerful Ideas'' children rnight develop as a result of experience with LOGO have 
been specified in slightly different ways by several researchers since they were Introdaced by 
Seyrnour Papert ([1972, 1986). 

P.H. Winston (1977), for exarnple views the possible powerful Ideas primarily In terms of 
prbgrarnmihg concepts rnbre broadly applied. He suggests applying the idea of state 
variables such as the turtle's orientation and pbsltlbn to the ternperatare In a rbbrn, applying 
the Idea bf cbntrbl variables like the rnbve and turn cbrnmahds given to the turtle to bttier 
cdhtrbl variables such as force, applying the idea of subgoalirig to other problems where a 
divide and cbnqaer strategy Is effective, and applying the Idea bf debugging tb other 
situatlbhs where the initial sbiutibh is Unlikely tb be perfect but where the primary parts sire 
basically correct. 

in contrast, FeUrzeig et al. {from Pea and Kuriand. 1984) suggest that programming 
changes thought as a result of the thinking skills ernplbyed, nbt just the specific 
prbgrammlng Ideas a prbgrarnrner must use. Fbr exarnple. rigbrbus thinking arid precise 
expressibri will deveibp because the computer requires programmers to use such skills. 
Facility with general heuristics such as planning, analbgy, and prbblern decbrnpbsitlbn will 
Increase since prbgrarnrnlrig prbvldes rnbdels bf them. Debugging will imprbve because it is 
central to the lr>teractive nature of programming. Self-consciousness and literacy about 
prbblern-sbiving prbcesses will be heightened since prbgrammlng prbvldes a vocabulary fbr 
discussing these pibcesses explicitly. In addition, general cbricepts bf procedures, variables, 
and hierarchy will deveibp simply because they are ideas encouraged in programming. Also, 
recognition of muitiple correct solutions, each with their own benefits, instead bf a single 
best answer shbuld deveibp since prbgrarnrnihg distinguishes process frbm product in that 
way. 
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bihh arid FIshir (1983) discriba pdwirful Ideas arising frorn styles of interaction with the 
cdmputer. The computer's Interactive feedback shoaid yield greater cognitive activity, the 
precision required by the Interpreter shouid yield complete speclflcatlbhs of ideas. The 
cbhslstehcy of feedback Is beneficial to the student; there Is no ambiguity and no prejudice. 
Lihh and Fisher aiso suggest that the chailenge of cbrnputer prbgrarnming leads to greater 
rnotivation and that the possibility of multiple sblutlbhs leads tb divergent thinking. 

Finally, Clements and Gullo (1984) discuss pbwerfui ideas as resulting frbm specific 
learning activities encountered in a LOGO ehvirbhmeht. They suggest that divergent thinking 
should Improve as. a result of students* inventing, constructing, and modifying their own 
projects: Since students reflect on their bwn thinking processes, they should make 
metacbghltlve advances. Considering errors and fixes should increase reflectivity. As 
abstract ideas become concrete in the LOGO rnicrbwbrid, cognitive develbprnent should be 
accelerated. The experience of giving spatial cbnimahds to the turtle should improve 
students' perspectlve-taldng ability when giving directions: 

In theory, other envirbhmehts could be equally gcod contexts for learning debugging and 
other high-ievel thinking slciiis. The advantage of LOGO over other envirbnrnents is that It Is 
a micrbwbrld with a limited set bf precisely f ;ecified units whIcH behave in a pricisily 
specified manner. The LOGO language provides an external representation at an 
appropriate level of access for eiernentary school children. Such a simplified context rnay 
make debugging more cdmprehenslbie than in the "real" world of ill-specified units and 
unpredlctabie behavior: The drearn Is that, once learned, thinking skiiis rnay be 
generalizable beybnd the LOGO '^•crbwbrld. 

1.2 the nightmare 

The transfer studies of LOGO are about as diverse as the ihterpretatibns bf what pbwerfui 
Ideas are available for learning, and the results of these evaluations seem contradictory: 
Some researchers claim tb dernbnstrate transfer, some fail tb demonstrate transfer, and 
others get mixed results even within the sarhe study. Gorman and Bourne (1983) and 
begelman et al. (1986) demonstrate transfer from LOGO to a rule-learning XBSk. Brown and 
Rood (1984) showed that LOGO and BASiC students irnprbved in prbblern-sblvlng ability, self- 
esteem, and Internalized Ibcus bf cbhtrol. Schwartz et al. (1984) repbrt that LOGO students 
increase motivation and cdgnitlon scores more than a control group, they also show that 
the LOGO group's rnath anxiety increases less and confidence decreases iess than the 
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cbhtroi group. But Pea (1983) failed to ehow trarisfer of planning skills, and Mc<3lHy et al: 
(1984) found that tOSO students demonstrated ho better skills In prdcedurallty arid 
debugging thin students who had no LOGO experience. At the same time, Clements and 
Guild (1984) and Clements (1985) reported that LOGO experience Improved divergent 
thinldng, refleotlvity, direction giving, and metacbgnltidnt but not cdghitlve development. 
Similarly, Gahlck (1984) found Improvements In school test scores, but not In spatial relations 
or combinatorial thinking after LOGO experience; Mohamad (1985) showed that LOGO 
students improved in spatial abiiity but hot quantitative ability, ability to syhthesize, or ahafytlc 
cbghltive style. 

This sampling does not exhaust the list of LOGO transfer studies (especially since many 
unsuccessful studies are undoubtediy not a part of the published or circulated literature); 
however, It Is representative enough for the purposes of this discussion. This section will 
attempt to explain the nightmarish results In terms of the basic requirements for successful 
transfer. The next section will then attempt to focus on the real possibility for achle\^ng 
triilsfer, based on the literature describing transfer of adult problem-solvtng skills. 

By definition, transfer rcfquires learning In the initial context and then exposure to a 
second context In which the learned knowledge and skills are relerant. the literature on 
tOGd transfer effects reports mixed results because of variation In the extent to which 
these two requirements are satisfied. In the LOGO domain, learning and transfer are usually 
studied in isolation, as If they were Independent: Researchers interested In detailed 
accbunts of learning frequently focus bh extremely simple. Ibw-level skills acquired over a 
few hours at most (see Roberts. 1984; Cuneo. 1985; and Campbell et al, 1985). Several 
researchers have exarnlhed the learning of higher level sidils: fbr example, kurlahd and Pea 
(1983) studied children's rhehtai models of recursion. Mawby (1984) evaluated students' 
understanding of variables and control structure, and McBride (1985) has begun to analyze 
LOGO prbgrammers develbplhg strategies and knowledge. However, these studies bf high- 
level skills are hot cbbrdihated with studies bf transfer, perhaps because they reveal 
students' lack of significant high-level skill acquisition. On the other hand, researchers 
interested In LOGO transfer frequently fall to document that students have Imprbved bh the 
desired skills within the LOGO domain prior to evaluating transfer. Of the ten transfer 
studies listed above, only three included assessments of learning (Pea and kurland. 1983; 
McGllly, 1984; and Garllck, 1984) Therefore, transfer of prbgrammlhg skill bh the bther 
studies: was assessed without a prior determihatibh that the skill was ever acquired In the 
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first pii^. Without an understanding of what LOGO students hive actually leimed (or not 
learned, as is often the case), transfer results are difficult to Interpret. Positive results may 
be diie to motlvitlonil fictors, and negative results may only Indicate that the required 
LOGO sidjjs were never learned. 

In addition to negiectlhg to document learning, most researchers studying transfer from 
LOGO fail to carefully consider the skills they expect students to learn or the skills required 
by their transfer tasks, so there Is often a rnlsrnatch between the transfer test and the 
training. Though there is considerable variety In choice of transfer tests, positive transfer 
results In the studies listed above tend to be on tasks Involving skills with figures similar to 
those used In LOGO, hot high-level thinking skills. The school tests used In Garllek's (1984) 
transfer test Involved questions about angles, distances^ and similar flgyres . Clements and 
GUIIo (1984) used only the fIgUral subtask of the Torrance test of creative thinking and 
measured reflectivity using the Matching Familiar Figures Test. LOGO studertts on?v 
Improved more oh the spatial part of the Developing Cognitive Abilities test In Mohamed i 
(1985) study. The skill similarity bccaslbhalty depends bh the particular version of LOGO 
used. Skills necessary for rule-learning tasks may be learned better In fl-LOGO, Used by 
Gorrhah and Bourne (1983), since It offers experience with anirnated sprites each of which 
has many Independent attributes Including color, number, heading, speed, and shape. 

On the other hand, negative results occur bh tasks whose relation to LOGO experience 
has not been specified, as well as usually not being documented. For example, Clements 
and Gullb (1984) sought transfer to classlflcatlbh and seriatlon tests of cognitive development 
and to metacognltlve tasks (see also Clerhehts, 1985). and Mohamed (1985) expected 
Improvement In ability to synthesize and analyze. Neither research group specified what 
relevant LOGO skills students had actually learned. Pea and Karland (1983) expected 
transfer of planning skills and Garlick (1984) expected transfer of cbmblhatbrlal thinking skills. 
Both assessed learning of LOGO content and general programming skills, but neither 
specified hbr assessed learning of the skills which were expected to transfer. McGllly et al: 
(1984) did document improvements In procedurallty and debugging skills prior to assessing 
transfer of those skills; however, they failed to specify how the learned skills were relevant 
to the transfer tasks (I.e., Tower of Haribi for assessing procedurallty and Masterrnlhd for 
assessing debugging). 

In addition to the problems of riot dbctirrieritirig iearhlhg arid not specifying the skills to be 
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inrned and trantfarad, ihera art many other methbdblogical prbblemi with moat of the 
bOGO atudlM. ao their reaulta are qaeatibnabje. For example, In many of the transfer 
studlei. performance of different treatment groupa was compared on a post-test wlthoat 
having any pre-teat comparison (e;g., begelman,1986; Gorman and Bdurne,1983; Clements 
and Guild, 1984; and Clements, 1985). Sbrne which did Include a pre-trat comparison did 
not use random asslgnrnent and had treatment groups with higher pre-test scores than the 
control groups (e g:, Schwartz et al., 1984 and Garlick, 1984). Others hid no control group 
at all (Gbrman and Bourne, 1983 and Brown and Rood, 1984). Flnaily, rnany of the reported 
transfer effects were actually small differences both In terms of the absolute numbers and In 
terms of the percentcge Irnprbvernent. (Schwartz et al. (1984) Is the most striking example.) 

thc« focus of this description has been on the principle that studies falling to demdhstrate 
transfet from LOGO cannot be offered as negative evidence of the possible cognltlva 
consequences of learning programming unless they first demonstrate that students actually 
learned the potehtlally transferable skills and then choose a transfer task on which those 
particular skills are useful, though there are problems In deterrnlning the leveto of usefulneM. 
bikewlse, studies that successfuliy demonstrate transfer cannot be offered as positive 
evidence Unless they also demonstrate that the effect is a result of students learning 
relevant skills during the LOGO experience. 

Researchers In other domains have paid careful attention to assuring learning and 
specifying relevant skills for transfer, the next sectldri will sUrnmarIze their results and the 
guidelines they offer for facilitating transferabie iearning and choosing appropriate transfer 
tasks, these guidelines will be Used to design a transfer study of LOGO debugging skiils. 

1.3 The reality 

The literature on transfer of prdblem-sdlvlrig skills has shown that when learning can be 
assured, transfer is iikely. Furthermore, researchers have begun to specify what must be 
learned for transfer to occur and to suggest ways to facilitate learning. Aisb. the transfer 
iiterature has shown that transfer is better to tasl(S for which the relevance of the learned 
skills can be recognized. Most Irnpbrtantly, these researchers are developing ways to decide 
what Is a relevant transfer task and what Is not. 
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1.3.1 SuccQSiful tmnifar of problirti-solving skills 

Bassolc and Hblybalc suggest that many of the transfer failures can attributed to 

Iniufffelem learning since few researchers provide more thin one trial trilnihg. In fact, 
Smith (1986) demonstrated that the provision of muitipie training trials Increased learning and 
transfer scores in a study of transfer between Tower of Hanoi Isomorphs. 

However, Bassok and Holyoak (1986) also suggest that overiearning is not sufficient to 
produce transfer. They view the difficulty of transfer as recognizing abstract structural 
commonalities despite surface differences. They claim that the acquisition of general 
schemas for problem types Improves transfer since recognition of a novel instance Is rnore 
likely. they suggest that the provision of multiple examples Is crucial to Ihductlbh of 
abstract schemas. 3ick and Hoiybak (1983) showed that transfer couid be enhanced on 
Isomorphic problems by givihg solvers a direct hint to apply the previoMSly used solution to 
the now probiem and that even greater enhancement resulted from requesting that solvers 
describe the similarity between problems (and presumably abstract the rele\mnt schema in 
the process). 

Bassbk and Hblybak (1986) compared transfer from algebra's arlthmetlc-prdgreMlor) 
probiems to physics' constant-acceleration problems and vice versa. They hypothesized ind 
fbiihd that there was more transfer from algebra to physics than in the other direction 
because the wider variety of arithmetic-progression problems yielded a more abstract schema 
which the subsequent physics probiems were more iikeiy to match. They described the 
general schema as productibh rules with abstract variables that can be readily matched by 
components of problems with the same structure, they cbhcluded that training will produce 
transfer to structurally Identical problems to the extent that the "abstract training In a 
problem schema is combined with practice in solving diverisie examples." Bassbk and 
Hbiyoaic (1986) also claimed that another facilitating factor In the physics and algebra 
dbmalhs was that students were taught the sfcilis directiy. 

i<btbvsky, Hayes, and Simon (1985) claim that the amount of learning Is also related to the 
source problem difficulty. The prbcessing ibad determines how much Information subjects 
learn on a first problem that they can then transfer to a second problem. They showed 
that Increased difficulty on Tower of Hanoi isomorphs decreased the amount of transfer. 
They attributed the decrease tb a iimitatibn bf wbrking membry capacity on the acquisition of 
transferable Ihfbrmatlbh. 
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Evin If ibitrict liimlhg dOM xnkB place in one domain, transfer to a second domain 
depend on the relevance of the learned stems to the new ddamlh. The diigihil form of this 
idea was ThbrndKe's (1913) identical elements theory. He suggested that learning of one 
stimulus reiponM link would only affect other such links If some of the factors were 
Identical elements, kleras and Bovair (1985) represented the elements of related procedures 
as production rules and demonstrated that there was more transfer between procedures 
when they had hidre Identical rules. However, Identical elements are difficult to identify and 
count. Other studies (mentioned above) show that It Is possible to transfer without Identical 
elements as long' as the skill Is represented abstractly enough, or production rules are 
genera j enough, that the relevance of the previously learned skills Is appare.it. 

Slngiey and Aiidersdh (1985) agree that transfer can be predict^ by the degrr^s of 
overlap between tasl^. They showed that learning a second text editing system took less 
time, fewer keystrdkes, and fewer errors despite a higher typing rate than learning the first. 
However, they attributed this sa^ngs to transfer of the high-level goal structure and the 
cbhceptua! mappings between text-editirtci commands and their actions. 

Daibey and tinn's (1984) research demonstrates the appllcatldn of this principle In the 
dbmalh of children's prbgrammlhg. They found more transfer to "robot tests" from "Spider 
World," a LOCab-IIke graphics mlcroworld, than from either BASIC or Type Attack, Though 
the cbmmahds are hot Identical In Spider World and the robot test, the actions of the 
commands and the goals of the task were quite similar. BASiC and type Attack did not 
have this similarity with the robot tests so the lack of transfer In these cbhditlbhs was not 
surprising. 

Ail of these studies highlight the need for transfer assessments to be based on detailed 
understanding of the skills learned In the tralnlrig phase and required In the transfer phase. 
Claims have been made that students will learn more trahsferable skliis when their learning 
Is abstract and when the memory load Is low. these principles were Incbrpbraied into the 
detiugu Instruction provided to the students In the dissertation study to maximize their 
potentia or learning transferable skliis. 
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, 1.3.2. Roaibing the dream of transfer from computer programming 

Newell and Simon (1972) suggested a threefold research agenda for studying human 
problem solving that Is consistent with the findings reported above: deveiop a detailed 
performance thebry» then address issues of learning, and finaiiy address issues of transfer. 
Simllirlyt Anderson (198^ stresses that his design of Intelligent computer tutors Is based on 
the premise that ''one needs to have a cognitive process model of the student If one Is 
going to be able to effectively tutor the student;'' Failure to base both instruction and 
assessment on a perfomiance theory has led to the distressing mixture of results In the 
LOGO literature. Researchers in the other problern-sbiving dornalns are attempting to follow 
this agenda and are documehtlhg learning ahd/br transfer. A search for transfer of skitis 
frorn prograrnming experience rnust follow a clear statement of what these skills are and an 
assessment of how well they are acquired during LOGO Instruction. 

My research agenda, therefore, inciudes three phases: 

• Detailed prbductlbh systerh analysis of debugging skilis in the LOGO domain. 

• Direct Instruction of those spis in Jw^ abstraction) with 
external support to reduce the working memory demands, and 

• /tesessmen^^^ skiil acquisition In the base domain and transfer to- domalhs 
requiring simiiar skilis. 

Anderson et al. (1985) have reported positive results from using a similar approach to design 
and assess the impact of computer tutors for te^aching both Lisp programming and gebmetfy 
skills. In contrast, rhy approach will focus bh designing prlncfpied instruction to be used by 
a human teacher In a typical classroom. 

The rest of this thesis is organized as follows. Chapter t describes a detalied task 
analysis to model good debugging skill {In the form of a computer simuiatibn) and 
sumrharlzes the resuits bf a piibt study designed to establish which of the model's skills 
sti dents learn in a "typicai" LOGO course (i.e., with hb expiicit instructibn in debugging); 
Chapter 3 describes the experimental design and the model-based Instruction and 
assessrrient techniques. Chapter 4 detaiis the students' sl<lii ievei in one LOCaO mini-course 
and the savings observed when they moved Into a second LOGO mlni-cburse. Chapter 5 
discusses the transfer of debugging skills from LOGO to ribh-prbgrammihg domains. Finally, 
Chapter 6 summarizes the findings and suggests ways to strengthen and apply them. 
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2. Analyzing the esmponents of debugging skill 

The debuggino instruction and transfer assessments dl<?cussed in this dissertatidh were 
based on a detiiled task analysis of bOGO debugging skills; The analysis was intended to 
capture, in trie form of a concrete model, the decision processes, knowledge, and sub-skills 
necessary for efficient debugging of LOGO graphics and llstorocessing programs with one or 
more semantic and/or syntactic bugs« 

The fbllbwirig sections describe the mbdei and Its applications for Instruction arid 
assessment. First, SectlcJn 2.1 is a gerieral characterization of the steps In the mbdei's 
debujjglrig process. Next, Section 2.2 is a description of the actual production system 
Implementatldri of the rt/ddel along with examples of It debugging fauity tOGO programs. 
Section 2.3 Is a report of a pilot study designed to determlrie which cbmpbrients of the 
model students learn in a typical LOGO course. The chapter wiil conclude In Section 2.4 
with a discussion of designing Instruction and assessmerit Iri accordance with the model as 
well as predicting transfer from debugging training In the context of LOGO programmlrig. 

2.1 A genaral model ; . 

The assumed debugging sliuailbri Is one in which the model has access to the program 
plan (the desired outcome), the buggy program, the output that the buggy prograrn 
produces, arid kribwiedge about LOGO, in the following analysis, we distlrigulsh betweeri 
the discmpancy and the bug. The former refers to the difference between the program plan 
and the program output. The latter refers to the erroriebus cbmpbrierit of the prbgram that 
caused the discreparicy. The goal of the debugging process Is to detect arid correct the 
discrepancy-causing bug. For exainpie, if the goal drawing corresponding to Figure la was 
a taller flower, then the discrepancy between the goal drawlrig and the prbgram output 
would be described Iri terms of the difference In size. The bug that caused the 
discrepancy would be the • 15 in the FD :D • 15 cbnimarid that draws the stern. 

Accordlrig to the model, there are five phases to the debugging process. The first phase 
establishes four subgoais that, wheri cbrripleted, reassert the top goal. The five phases are: 

1. Prbgram Evaluation. Ruri the prbgram. Compare the prograrn plan and the 
program output. If they do ribt rriatch perfectly, then identify the bug, represent 
the program, locate the bug, arid correct the bug. 
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2. Bug IdehtlffcatlbhV Generate a descrlpHon of the 
program plan and jhe program omput^ Based on the discrepancy description, 
propme spMinc types of bugs thaj mighty be responsible for the discrepancy. 
Where ffiultlple poMlbllh^^^ exist, do farther discrepancy description and bug 
proposal. When only one possibility remains, examine the program output to 
Identify the specific bug. 



• In Its purest form, the discrepancy descrlotlon makes ho reference to the 
fact that the faulty output is program-generated. That Is. the discrepancies 
are c^haracte^lMd enti^^^^^ their static features.^ Table 1 lists 

the most common types oj discrepancy encountered when debugging LOGO 
graphics and^ list-processing p^^^^^ The quotations presented In the 

second coipmn are representative comments from children In this study 
about the type of discrepancy sh^ In the first column. Note that one 
possibie outcome of the bug Identification step Is knowing that the plan 
and output are not Identical but being unable to describe the mismatch. 
However, in the case of syntax errors, the error message always provides a 
description of the discrepancy for the user (though the user may Ignore It). 

i Given the description of the discrepancy, the model makes Inferences about 
which specific .program cbmpbhents are capable of generating that type of 
discrepancy. _ The third cdlumh In Table 1 suggests some of the pbssible 
mappings. For example. If the dlscrepancy^Js spread, then It is likely to 
be caused by turhlrig the wong angle or mov|^^ In 
addltlbh to proposing these gene^ of prbgrarnrning errqra. the model 

has a^ set of rules which propose farther discrepancy description to 
drscrimlnate between niaK[pie posslW^^^^^ \/Vhen only one possibility 
remains, the modei exanilnes the program output to determine the specific 
^og. The mo6e[ may need \^ through discrepancy descHptlpn and 

bag proposal several times before a specific program command Is Identified 
as the bug (see the fourth column in Table 1), However, the result of this 
complex processirLa is a narrower search for the bug (e.g., "right here - It 
shouldn't be left 90 - it should be right 90 I think"). 

3. Program Represehtatlbh. Represent the structure bf the prbgram to Investigate 
the probable Ibcatlbh bf the buggy cbmmand !ri the prbgram listing. 

• Knowledge of the program s structure may be the result of having written 
the program br bf assuming that prbgrams fbr certain types of plans will 
be structured In characteristic way^^^ For example, the model may be 
given knowledge that the prbgram has a repeat structure because the user 
wrote the prograni or becaase the user observes that a picture is 
composed of several Identical figures (typically programmed using a 
REPEAT statement), knowing that the bug is located within a REPEAT 



It Is possible that discrepancy descrlpiidns might ihcliide temporal Ihfbrmalrbri. because rri bur jDrbcedure, the 
?^iL<^ _^l*Cjies as the prbgrarri's bufpijl is dynamicaliy generated. On the computer we used. Figure la would 
take about S^econds to draw: Also. Jbi^ list-prbcessirig output, the temporal order of different porirons of the 
butput Is preserved by the listing on the screen. 



EKLC 



2B 



transfer of Debugging Sklil ^7 



staternent narrows the search jn |he program Jsting. In the case of syntax 
errbi^i the errb^ message gJves the aser Information about which procedure 
contains the bug. 

4. Bug bocatlb^^^ gathered In the last two phases, examine the 
program In order to locate the alleged bug. 

• The efficiericy of thb bug location process depends bh the butcbme bf the 
bug Identification and program representatlbh processes. At best, the 
model searches for a perfectly specified bug (bbth the buggy cbrnrnand 
and Its arguments are specified^ In a highly cbnstralned set of possible bug 
Ipcatlbns. At worst, the mbdel must perfbrm a step-by-step exarnlnatlbn of 
the prbgram because It has no khbwiedge of the bug's Identity and no 
cues abbut Its Ibcatlbh. 

5. Bug CbrrectLbh. Examine the program plan [o deternilne the appropriate 

cblrectlbh. Replace the bug with the correction In me program listing and then 
reevaluate the prbgram. 

• This reevalutatlbh is slightly different frbrn the Initial test In that the model 
knows a change Has Just been made. It first determines whether the 
cbrrectlbh fixed the briglnal problem^ If the correction vyorked. the model 
will determihe whether there are any more bugs to fix; . otherwise. It will 
debujj the cbrrectlbn before proceeding. ^ 



Insert Table 1 abbut here 



2.2 A production system specification 

in order to specify the model uhambigubusly arid tb derribristrate Its sufficiency for 
debugging LOGO prbgrams. It was Implementod In C3RAPES, a goal-restricted prbductlbri 
system (Sauers and Farrell. 1982). The GRAPES model consists of a set of rules, called 
productions, which specify the action to be t;ikeh If certain cbriditibris exist. The conditions 
Include the gbal the model is trying to achieve and the information currently available in 
working memory (the set of known facts). A prbductlbri is selected and executed only when 
the appropriate conditions exist; thus the current state of the ehvirbrinierit determiries which 
actions will be pertormed. The actlbris iriclude updating or adding to both working memory 
arid goal memory. 

The 84 prbductioris iri bur GRAPES rnbdel represent our task analysis of debugging skills 
in the LOGO context. The model's gbals represent the steps iri the debugging process; the 
heuristics It employs represent general and specific search strategies used for efflclerit 
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debuggino; and the operators it invokes represent sub-skills which are essehtlah but not 
cihtril, to the debugging process (e;g.. editing skills), the foHowing sections, will describe 
these three components of the model in detail. These deserlptlbhs will be followed by 
demonstratlbhs of how the goals, heuristics, and operators work together to debug faulty 
LOGO prbgrarns. 

2.2.1. Goals direct the soiutlon 

The debugging rnbders goal structure cbrrespbnds to the five phases described In the 
overview. A goai tree Is shown In Figure 3. the system has a set of productions for each 
goal to represent the different responses a debugger would have to the same goil In 
different situations. The "situations" are represented by the current cbntenta of the s^tem's 
working memory. Prbductibhs with test and evataate goais start the system and evaluate 
the success of each debugging attempt (I.e., the match between the prbgrarn pian and the 
program output). The describe and propose goals correspbrid to the bug IdehtSflcatlbh 
phase; they satisfy the prbductibhs that describe the discrepancy between the program plan 
and the program's buggy output and that propose pbsslbie bi^ and ways to discrirninate 
among them. Represent and specify correspond to the program jripresematlbn p 
productions with these goals look for structural cues tb the bug's iboiion so that find, 
interpret, and check can actually isolate the bug using whatever cues they have about the 
bug's identity and Ibcatlbh. Firialiy, the cfiange and replace goais correspond to the bug 
correction phase; they fire productions tfiat Identify the apprbprlate correctlbh and change 
the prbgram listing accordingly. The fuii production system Is presented In Appendix I. 

insert Figure 3 about here 



2.2.2: Heuristics narrow the search 

The system has two sets of debugging heuristics, one set for identifying the bug and one 
set for representing the location of the bug in the prbgram. Using both sets of heuristics 
riarrbws the search for the bug substantially. Heuristics for identifying the bug cbrrespbrid 
to the mappings between observed discrepancies and potential bugs (listed in table 1). 
These heuristics are most useful when there Is mbre than one type of bug which can lead 
tb a particular type of discrepancy. In this case the heuristic ihcludes irifbrmatlbn fbr 
distlrigulshlhg between them. Fbr example, if the discrepancy has been initially identified as 
spread, then the model will request Irifbrmatibri about orientation because it has the 



EKLC 



Transfer of Oebugglhg Skill 

. khbwlodge that discrepahclas dascrlbed as both spread and orientation mast have been 
eaused by an angle bug whereas those discribed only as spread discrepancies must have 
been caused by a distance bug. 

Heuristics for locating the bug involve Knowledge of program structure types. For 
example, if the program is identified as having subprogram structure, the model would ask 
for infbrmatibh about which subprogram was likely to contain the error and It would confine 
its search to that subprogram unless no bug could be Ideated there. If hb subprbgram cue 
is available, the model will seek other structural cues, such as location within a REPEAT or 
IF statement or location after a particular cbmmahd. Fbr example, If the user can Identify 
a correct command which was executed before the bug occurred, the model will use that 
Cbmmahd as a marker and begin its search after that command. 

2.2.3. Operators process information and produce beha^or 

According to our model, the debuggirig prbcess uses 11 operators, or sub-skills, to 
process Infbrmatibh available to the system. These operators afp^ciMd by prbductlbns 
when It is necessary tb process Ihformatibh from one or more sburcw? or to take specific 
actions. Four sources of Information are available to the bperatbrs at all. limes: the 
program plan, the prbgram output, the program listing, and knowledge of the programming 
language. In addition, ah operator may use Ibcal Ihfbrmatlbri contained In a working 
memory element, dperatbrs may also add Information to working memory. 

There are two classes of bperatbrs: a) thbse that cbrrespbrid tb inspection of the buggy 
butput and/br the plan, and bj those that correspond to marieuverihg in the LOGO 
environment. The latter set bf bperatbrs are autbmatlcally executed by the model, but the 
fbrmer set are not. Instead, their operation is simulated by the user bf the system. 
Essehtlaliy, the user cbmpares the buggy output and the program plan for the system, 
inputting judgments bf whether a prbgram did what It should have, estimating angles and 
distances, reading error messages from the LOGO screen, etc. 

A brief description bf each bperatbr fbllbws. Wbrking rnembry elements are presented In 
parentheses and Italics are used for variables represehtihg the system user's Input; Fbr 
example, the wbrklhg rnembry element (discrepancy response yes) might actually be 
instantiated as (discrepancy brientatibn yes). 
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• Tha MATCH optirttor is caiiid to procesi informatlbh from the prbgrim plan arid 
tAa_ pro9r![n output and Input It9_ judgment about the match between the twa 
The a^em quntlona the user, "Old the outcome match the plan?," or '^Dld the 
simulation match the j^lan?," and expects a yes or ho response. A yes 
respohii cauaei the element (match yes) to enter working mernory; likewise^ a 
no response yields the element (match no). If a command has just been 
changed, the ^ysterrTs querj^ls, "Did the correction fix the problem?" and the 
resulting working memory element Is {tlx yes) or (fix no). 

• CONTRAST^ Is a dlscr^^^^ operator that processes Infprmatiph from 
the program plan and the program output The system flrsl asks about the type 
of dlsc^repancy (graphics or jlsts^ semantics or syntax). Then it asks a more 
focused question about the panicular discrepancy and gives a list of possibilities. 
For example, the system asks, "What is the discrepancy between the plan and 
outcome?" and requests one of the following answers: brlehtatlbh, size, spread, 
Idcatidh, extent, or ? for graphic sehiahtrc discrepancies, A working memory 
elemiht bLJhe_fbmi (discrepancy response yes) Is then added to working 
memory, CONTRAST can also Indicate whether a specified discrepancy exists or 
hot. If a wbrklhg memory element such as (description must be about size) 
exists, then the system's query Is, "Is size discrepant?" A yes or ho response 
Is expected ahd the resultlhg wbrklhg memory element Is of the form 
(discrepancy size response). 

• EXAMINE Is a bug proposal^ operajor w^^ seeks specific information about the 
bug usihg LOGO knowledge Jo guide the processing of' brfy n iiUHiti from the 
program otitconie. The systeni^ asks t^he user a question stRf^ w f Wffw Is the 
discrepant angle on the outcome?" and labels the user's i(HMUHS> ai.tlw bug, 
(the bug could be (RT (120))) for example. 

• INTERPRET Is a bug location operator which simulates the effect of the current 
command using LOGO knowledge. The system reminds the user of the functfbh 
of a command, then asks whether that was the appropriate cbmmahd to use, 
then asks jvhether each bf the argumehts Is correct. These qtiestlohs are calls 
tb the MATCH bperatbr described above. 



• GENERATE is a bijg cbrrectjoh operator wh[^ uses tOGb knowiedge to 

determihe jvhat command would be necessa^ to accomplish the desired effect. 
The system asks the user^ "What should the command bug have been?." or 
"W]iat command should be Inserted?," and creates a new wbrklhg membfy 
element from the user's response, (the correctibh is (RT 90)) fbr example. 

• The other six operators, basically editing bperatcrs. are associated with jDhyslcal 
action In the LOGO ehvirohmeht. The model automatically carries but these 
bperatlbhs bh Its representatlbh bf the LOGO ehvirbnment. while hbtifihg the user 
that Jt Is dblhg so. These bperatbrs ihciude RUNriihg the prbgram, ENTERIng 
the editor tb view the prbgram^ jistjng, Skipping to a particular location In the 
prbgram Jlstihg. REAbIng a command, DELETing a command, and INSERTing a 
command. 
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2.2.4. Modeling solution strategies 

the moders solution to a debugging probiem depends bri the ambuht of Information 
gathered about the debugging situation to guide the search for the bug and the accuracy of 
the Ihi^ut risulllhg from the user-simulated bperatbrs. in this section, we contrast two 
debugging examples with different amounts of Knowledge. The more Rhbwiedge Input tb the 
model, the narrbwer the search for the bug. Appendix 11 provides additional examples to 
show how the model recovers from being given Ihcbrrect Input and tb dernbnstrate its 
flexibility in deaiing with a wide variety of debugging situations. 

This example is extracted from one of the actual graphics debugging tests used In the 
dissertatlbn study. The desired butcbrne was for LQQO to draw a corn field (Figure 4a). 
the program's outcome was, however, discrepant from the plan (Figure 4b) because there 
was a bug in the program. The relevant portion of the test program Is shown In Figure 4c. 
the two traces discussed In this sectlbn (Tables 2 and 3) differ in the amount of 
information about the bug's Identity and location, the goal numbers In the discusslbn refer 
tb these traces. 

Insert Figure 4 about here 



In the first trace (Tabie 2). we simulate a situation In which the debugger Is a very 
knowledgeable user. The model Is provided with a ibt of Infbrrnatibn about both the 
discrepancy and the program, the Information, provided In response tb the bperatbrSi lis 
rnarked by -> bh the right-hand side of the trace. Here the user classifies the probiem as 
graphics without an error message (gbal-2) and then identifies the discrepancy type as 
"wrbngpart" (goai-6) since the outcome has four ears of corn instead of fbur stalks. The 
model then prbpbses that the wrbng subprbgrarn has been called (goal-7). It asks the user 
the name of the wrong subprogram. The user respbnds that it Is caiied eiSRN. This 
user's clever discrepancy description led to a very specific proposal of the bug: bther 
descrlptlbhs. e.jj. "misslrigpart", would have worked also, though perhaps not as efficiently. 

When prompted about the subprogram structure of the program, the User responds with a 
? (gbal-3), Indicating uncertainty about the structure and about whether the bug is In a 
substructure, the user then Indicates that the bug is prbbably in a REPEAt staternent 
(goals 8-9) since there are four identical figures. Knowing that the bug is In a REPEAT 
statement plus that It is prbbabiy a wrong subprogram call to CORN allows the model to 
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And tHl bug directly (gdlR). The model asks the user whether CORN should be changed, 
deleted, or have another command Inserted before It (gdil-S^ The user Indicates that a 
change la rieceiiiry so the model requests the user to input the correction (goal-llj. it 
rtiakei the correction and then instructs the user to rerun the program to check the 
correction (goals 12-14). The correction Is accurate so the mpdel asks whether there are 
any more problems (g6ai-l5). Since there are not, debugging Is complete. Figure 5 shows 
the goal tree generated during this debugging cycle. 

Insert fable 2 about here 



Insert Figure 5 about here 



The second trace, In Table 3. Illustrates the moders behavior whert the user knows 
nothing about the discrepancy between the plan and the outcome and nothing about the 
structure of the program. The User runs the prdgram arid kribws that a discrepancy exists 
(top-goal). When asl<ed about the type of discrepancy, the user knovMLthat It Is a graphics 
problem without an error message (gbai-2). The user re^nda^ta ttw*: specific type of 

graphics discrepancy with a ? so the model cahhbt propoV^ ft^; bc^ IdenM^ (go^ 6-7); 

_ " "^^^ _ ' ' • 

When asked about the program's structure, the user again answers wtth e ? so model 

tries to get Informatibh about REPEAT arid If statements arid other cbfmrwnds »m mtght 

be useful as markers (goals 3,8-10). Again the user responds negatively so the model riiust 

search for the bujg by Iterating through the prbgrarn listing (goals 4,11-28). For each 

command, the model defines the cbjninarid arid asks the user whether It was the 

appropriate cbmmarid to use: Then It asks the user to check each argument. The first 

command in the REPEAT stateriierit Is theri deterrriiried by the user to be the bug (goai-29). 

the model aslcs whether a change, a deietibh. or ah insertion would be appropriate arid the 

user decides to make a charige (goai-5). The model requests a correctibh which, once it is 

input by the user, will be substituted iritb the prbgrarn listirig iri the place of the bug (goals 

30-31). Finally, the model directs the user to rerun the program to be sure the cbrrectlbri 

was accurate (goals 32-33). Slrice it was and no other discrepancies existed (goals 34-35). 

the debugging episode was complete. 

irisert Table 3 about here 
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Figure 6 comparee schematle verelbha of the goal trees for the two tracea. The contraata 
between the modei'a behavior in the high- and low-lnfoririatlbn sltuatlbhi are strHclngi the 
former required only 17 production firings, while the latter required S2. With respect to the 
complexity of the sub-goal tree, shown In Figure 6, we see the same kind of difference; 16 
sub-goals va 35. the high-lnformatlon simulation represents the Ideal case In which the bug 
is completely specified and its location is icnbwn. The system's goals and heuristics were 
used efficiently to harrow the search for the bug. In the Ibw-lnfbrmatlbh situation, little use 
is made of the describe, propose, represent, and specify goals so none of the heuristics for 
narrowing the debugger's search are used and debugging proceeds by brute force, one 
command at a time. Most of the extra production firings arid sub-goals result from this 
serial search. 

Insert Figure 6 about here 



For the purpose of this slmuiation. we chose an example where the bug was close to the 
beglrihirig of the program and assumed that the interpret operator correctly identified the 
bug. if this had not been the case, the difference between ftm^ tma traces would have 
been even more striking because Ibriger and/or repeated debugging -cfttmr.watM have been 
necessary. Interested readers should peruse the traces iri Aj Bf»K Mx * It to ger n man pbbal 
picture bf hbw the model works. '^K' ■ ■ ' 

2.3 Students' llrnited debugging skills 

A pilot study (Carver and Kiahr, 1986) used the formal task analysis of debugging as a 
context for assessing how much bf the debugging skill specified by the mbdei children 
actually iearn in a guided discovery LOGO graphics envirdnment. A guided discovery 
erivlrbrimerit Is brie Iri which the instructbr introduces new LOGO concepts and may give 
project Ideas for trying them, but theri the studerits are free to create projects bf their bwri 
chbice. the course was designed to assess the acquisition of both debuggrng skills, the 
model's goals. Heuristics, and debugging bperatbrs (MATCH, eONTRASt, and EXAI^iNEj; 
and conventional' LOGO sub-skills (the mbders gerieral bperatbrs). Debuggirig skills were 
evaluated at three times during the LOCaO course by asking students to describe the 
prbbleril with the butput bf several programs and then asking ihem to view and debug the 
programs. VV?> assessed the other bperatbrs assumed by the mbdel at three times alsb. 
The ability to lIsJTERPRET commands to predict the behavior they would cause was 
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asiissed using a papsr and pencil task; students were given programs and asked to draw 
their ootcomM. We ased a tartie target game to assess students' ability to GENERATE 
cbftimahdi to ciuia specific behavior: students had to give the turtle one turn command 
and one move command to make It reach a target, the ability to maneuver within the 
tOQQ programming environment (RON. ENTER, SKiP, READ, DEtETE. and INSERT) was 
tested using a program editing task; students were given a hard-copy of a program with 
changes marked on It and they were asked to do the editing. 

Nine children (5 females arid 4 males) ranging in age from 7;1 to 8;9 participated In the 
study. They were recruited from the communities surrounding Carnegie-Mellon University by 
advertising a free LOGO course. None of the children knew any computer programming 
languages, but most of them had limited experience either with computer games or with 
programmed Instruction. Eight of the subjects came In pairs and one came Indlviduajry to 
12 two-hour LOGO classes over a three week period during the summer All lessons were 
tatight by a 24-year-oid experimenter who was an experienced LOGO instructor. the 
Instruction used an APPLE* lie computer with terrapin LOGO. Skills In cbmrnarid 
Interpretation, command generation, maneuvering within the LOGO environment, and 
debugging were tested three times during the course. 

After 24 hours of tOGO expeience, which is as much or more experience than was 
provided in 9 of the 10 studies mentioned in Chapter 1 (Clements. 1985), bur subjects had 
not developed effective debugging strategies, in fact, they rarely debugged programs when 
they were not required to do so. Figure 3 depicts the general results in terms of the 
proposed debugging model: the boxes around the goal names indicate the parts of the 
model with which the children had difficulty. In the test phase, children were able to 
MATCH the goal drawing arid the program output to determine whether or not they were the 
same. However, they may have found debugging tedious because, accbrdirig to our mbdel. 
they lacked kribwiedge bf the discrepancy-bug mappings and heuristics for locating the bug 
in the program ilstirig. When forced tb cbri^rrierit bri the nature bf the prbblem, they 
demonstrated limited ability to describe the discrepancy. Also, their propose phase was hot 
useful because bf their inability to discriminate between potential bugs and their poor 
EXAMINE operator. Iri additibri tb this restrlctibri bri debuggirig, the chiidren had few cues 
to represent the program {since they had riot writteri it). Everi wheri clues had beeri 
meritibried, they were seldom used to narrow (he search for the bug, so children had to 
find arid charige the bug usirig step-by-step examiriatibri of the program. Despite good 
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skllli In mahiuvirlhg within thi LOGO ehvlrbhrnent (RUN, ENTER, SKIP, etc:), the seriai 
search was tedious because of the children's poorly developed skills at iNTERPRETIng 
cbmmarids. However, children were generally good at QENEFlATIng the command to correct 
the bug once It had been located. Though the chlldreri In Carver arid Klahr's pilot study 
did learn some tOdld skills, they did not learn the goal structure, the heuristics, or the key 
operators that the rriodel uses for effective debugging. Also, In their own debugging, they 
tended to skip the phases before looking at the program so they were usually left to serial 
search when they debugged at all. 

The debugging model assumes that the child has already made the declslbh to debug a 
prbgram. rather than to simply abaMibri it arid start over. However, Carver and klahr's 
(1986) pilot study as well as many educators' Irifbrrnal bbservatlbri (Papert. 1980) Is that 
chlldreri prefer tb restart rather thari to debug. We believe that In most cases children 
don't debug because they dbh*t know hbw tb. There are three reasons why they fall to 
acquire this skill: a) Debugging Is a complex skill; b) It requires extra meinbry capacity; 
arid c) It Is rarely taught directly. Belbw. we elaborate each of these stumbling blocks In 
relation to the model and to the relevant literature. * 

2.3.1. bebuijjglrijj is a cbmplex skill 

Debugglrig's complexity is obvious from the large number of goM; he wSHcs , arid 
operators necessary tb describe bur riibdel. The bug identification and bog. focation 
productions represent a minimal set of heuristics fbr finding bugs; vvlthbtit these search 
shortcuts, the riibdel 's debuggirig is laborious. V\/ell-devejoped operators are essential for 
accurately comparlrig the actual output with the gbal output arid for Iriteractlrig with the 
tOGb system; without such operators, the model makes frequent errbrs arid requires mariy 
cycles tb cbrrectly debug a prbgrarii. 

Research on adult programming skills has shown siriiiiar difficulties ambrig ribvlces: 
Studies with adults have shown that novice programmers fail to use the goal structure of a 
program as an aid tb bug Isblatlbri. Fbr exariiple. Jeffries' (1982) study of expert and 
ribvice Pascal programmers showed that grbwing expertise involved deveibpirig a hierarchical 
represeritatlbri bf prbgrariis. (She also found that experts had accummulated a set of 
familiar patterns that they used tb relate flaws Iri the output tb potential bugs. Our model 
represerits such knowledge in the propose prbductlbris.) Spbhrer, Sblbway, arid Pbpe (1985) 
make the Iriterestlrig suggestion that failure to maintain the appropriate goal hierarchy ("goal 
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dropout'' and '"merged goili') Is a cbmmbn source of bugs. Atwood and Ramsey (1978) 
fband that debugging waa difficult for adult novice FORTRAN progrimmeri because they 
lacked useful heuristics for seeking cues from faulty output which could narrow their search 
for the bug. 

Also, Llttmah (1986) has shown that even expert programrners do not use an effective 
search-nari'owing strategy (*as-needed/ In Llttmah 's terms) when attempting to modify large 
programs. Rather, the experts who used a brute force strategy (^systematic.* In LIttmah's 
terms) succeeded more often than those using the as-needed strategy, the success of the 
systematic strategy depends heavily on a good interpret operator and a program of 
manageable size. Good ts-heeded strategies, similar to the cue-directed search the 
debugging model uses, would be necessary for large software projects. However, the 
experts who did use the as-needed search in tlttman*s study did not demonstrate good 
strategies; they had difficulty determining what was "heeded.'' 

Although goal structures play an Important role in adult programming, a hierarchljal 
conceptualization of the solution to a programming problem Is very rare at the low level of 
programming skill typically reached by children, which Is perhaps why:^1Mir• do poorly. For 
example. In Pea's (1983) study, children were able to debug syntax eriora-^fectlveiy but 
were not able to iocate semantic bugs such as mlsdrdered commands. Part of their 
difficulty was a result of their tendency to approach programs as long chains of direct 
commands rather than as hierarchical structures. Nevertheless, because we wanted our 
model to represent an efficient debug-fdr, we did Include the capacity to effectively use 
knowledge about the goal structure of the buggy program (in addition to a iast-resort brute 
force strategy), if the responses to the represent operators Indicate that the program has 
a specific structure and that the bug appears to be local to a particular component of that 
structure, then the model Immediately constrains its search to that ideation. 

2.3.2. Debugging sj<ilis require extra capacity 

Development and use of debugging skills requires memory capacity sufficient to keep tracl< 
of available cues. Even though most of the productions in our model require only two or 
three working memory elements as conditions, this is more than novices can handle in 
addition to the already heavy memory load imposed by learning to program. Aiso. the 
context in which students typically experience a heed for debugging is one in which their 
attention is directed toward the probiern at hand: getting the program right, this focus 
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leaves them little capacity to learn about the debugging process Itself. According to 
kotovslcy et ai. (198S), this minimal avillibli cipaclty would result in minima t iearning and 
therefore minlmai transfer. Anderson and Jeffries (1985) suggest that many of the errors 
adult novices make In computer programming were the result of working memory faliure. 
Children may be even more susceptible to this difficulty. 

in fact, Pea and Kuriand (1984) suggest a a muiti-stage characterization of the acquisition 
of programmlhg skills that emphasizes the capacity Issue. The beginner Is simpiy a ''code 
generator'' who focuses on individual commands rather than developing a structured 
program. Next, the student begins to think In terms of higher ievei units, becoming a 
"program generator'' who can create and debug complex programs. Finally, the student 
becomes sufficiently familiar with the language that he can distance hirnself from the coding 
processes to consider the general problem-sdlvlhg aspects of programming such as 
elegance, efficiency, and optimization; he has become a ^software developer" Only at this 
level can he deal with high-level thinking skills to the extent that transfer would be possible. 
Much evidence suggests that even after 20 to 30 hours of Ihstructldh, most children are 
barely out of the first stage (Carver and Klahr, 1986; Pea and Wiiriiii». 1983; Mawby 1984); 
they are stiii struggling to acquire the basic LOGO operators. lllitK^NSfl^^ showed 
that even a ^mall sample of tOGO teachers failed to debug complex LOOO pfograms. 

Focusing on the compiexlty of debugging and the need for extra capacity to learn It has 
shaped the agenda for research on novice programming; Most researchers study novice 
difficuitles so that Instruction can be improved and students can reach the high-level skills 
more quickly. Mayer (1981) viewed the novice's problem as one of concept 
meaninglessness, so he studied techniques for making computer concepts more meaningful. 
Also, Spbhrer and Soloway (1986) have been devoidpihg a process model of novice bug 
generation. Pea and Kuriand (1983), McBrlde (1985). and Mawby (1984) have already been 
mentioned as examples of research on children's poor understanding of hlgh-ievei 
programming concepts. The focus of our model has been on identifying good strategies so 
that the goals for instruction and assessment will be clear. 
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2.3.3. Debugging Is not directly taught 

Although debugging la a complex cognitive ak^l whose acquisition requires substantial 
cognitive capacity, It may not be difficult to learn. The important question of whether or not 
children can be directly taught to debug programs is cbmpletely open because debugging is 
rawly an explicit part of a LOGO currfcurum, that is, although chlldreri in a typical LOGO 
course rnay get exposed to debugging in the process of getting their prbgrarns to work, 
they do hot get explicit instructibn in detecting discrepahcles, Inferring error sources from 
discrepancy descriptions, and so on. 

Gugerty and Olson (1986) reported that In a debugging situatlbh expert prbjjrammers are 
better able to comprehend code and generate high quality hypotheses about possible bugs 
than are novices. They suggest that these skills must develop as a result of experience 
since debugging requires tremendous knowledge. They Imply that being a good programmer 
Is a prerequisite for being a good debugger. 

However, kessier (1986) has shown that It Is pbsslble for tiSP.^udenta to learn to debug 
simple functions before having experience writing thern. It rnay fW- Jinlflli,. then, to teach 
good debugging skills to novice prograrnmers so that the frequent tni991lliV»ptVie will not 
be siicM a stumbling block to their learning process. 

2.4 Applications of the model 

By doing a detailed task analysis of debugging skills, we have been rble to specify the 
component processes, heuristics, and sub-skills that programmers must learn In order to 
debug well. We have also established that students do not learn the central cornponents of 
the model spontaneously. Similar difficulties with debugging have also been demonstrated Iri 
the adult programming literature. The usefulness of the model does not end with 
characterizing difficulties, however. Our perfbrrnance theory can be used as the basis for 
designing curriculum to teach components of the debugging model as well as for designing 
transfer tests and measures on which acquisition of the model's skills will be demonstrated. 

2.4.1. Designing ihstruction 

The objective of debugging instruction Is to train students to use the model's debugging 
procedure, especially the initial phases where cues to the bug's identity and the prbgrarn 
structure are gathered to narrow the search for the bug. The rhodel's goal structure could 
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be introduced as a atep-by-step procedure, after a jittie rewording for elementary studehta. 
Simllarty, itudenti could be taught to ask themselves the sarne questions the rnodel asks 
the uaer. the specinc heuristics the model uses to map discrepancies onto likely bugs and 
to focus search on particular parts of the prograrn could be taught to students directly. 
Such Instruction would decompose the complex debugging task Into sirnple steps 
comprehensible by novice programmers. 

In addition to teaching the cdhtent of the model, Ihstructlbh must take the students' 
memory capacity Into account. Providing memory aids such as posters with cdmmahds, 
debugging stepsi and discrepahcy-bug mappings would help to reduce the rnernbry load to a 
manageable amount. 

2.4.2. Predicting and assessing transfer 

Given that students can, In fact, learn the debugging stdlls we have modeled, they should 
be able to use them In other contexts In which they are recognizably appropriate. Students 
who learn debugging In the context of a tOQO graphics course would be likely to recognize 
LOGO llst-prdcesslng as a domain where their debugging skills wm]id.:te useful, and vice 
versa. Our model shows that the goal structure Is Identical for debugging graphics and llst- 
prdcesslhg prbjgrams, as are several of the discrepancy-bug rnappings (wrongpart, extrapart, 
misslngpart, howto, whattd, and novalue In bur model) arid the prbgrarn structure cues. In 
addltlbri. the sub-skills required by the debugging process are similar Ir? these twb LOGO 
domains. 

Mbre gerieraliy, the debugging skills students learn In the context of LOGO programmlhg 
could be recognizably useful in ribri-prbgrammlrig tasks, particularly those requiring extensive 
search. In a non-programming domain, the sub-skills arid specific discreparicy-bug rriapplrigs 
are ribt likely tb be similar to those used In programming. However, the five pnase goal 
structure would be similar tb bther debuggirig situatioris as ibrig as the desired outcome, 
buggy directions, and buggy outcome are available to the sblver. Alsb, If the buggy 
dlrectlbris are structured Iri ways similar tb LOGO prograrns. :hen the program structure cues 
should also b*3 similar. The transfer tests used fbr the research discussed in this 
dissertatlbri were designed with these criteria for similarity in mirid. 

The measures used tb assess debuggirig skill were also based on the rnodel. When 
using the model, greater knowledge input results iri riarrower search (fewer gbals); 
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developing debugging skill should therefore result In decreased debugging time. If all the 
knowledge Input to the model it accurate, the bug can be located and corrected al! Iri one 
cycle through the model (from the Initial goal to test the program lo the final retest goal). 
Developing accuracy In debugging should therefore result In fewer debugging cycles neaded 
to locate and correct bugs, these moasures of speed and efficiency, along with sbrhe 
qualitative characterizations of debugging strategies, will \oq the core of our transfer analysis. 

The details of the modei-based Instruction and assessment wili be described In Chapter 3. 
The primary goal of the Investigation was to assess the extent to which students trained to 
debug one type of LOGO programs could transfer their skills to a second type of LOGO 
programs and to debugging of non-computer directions. 
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3. Designing model-based instruGtion and assessment 

The formal task analyaja of LOGO debugging skills presented In Chapter 2 provides a 
basis for dataliad instruction and assessment of those skills, this eJ<perlment was designed 
to address several Issues. The first goal was to discover whether students can learn the 
debugging skills used by the model when those skills are taught directly. The second goal 
was to derndnstrate tliat debugging sklllSt once learned, are transferable to tasks requiring 
similar skills. These Issues were addressed In the context of a 5G hour bOQO graphics and 
list-processing course taught to 22 8- to 11-year-oid children In a Montessdrl school over a 6 
mdhth period. This chapter will discuss the experimental design, the reje\^nt rnethodologlcal 
issues, the instructional techniques, and finally the assessment techniques. 

3.1 A ebmbihatibh of transfer designs 

The design of this study Is a combination of two common transfer designs. A savings 
design was used to assess the learrilhg of prbgrammihg. debugging, and editing sklils In 
one L0Q0 mini-course (graphics or list-processing) and the trans^ of those sklils to the 
other miril-cburse. A pre-test/pbst-test design was used to ,assm the trm^r of debugging 
skill learned In a LOGO envirohmeht to ribri-cbmputer debugging; tajSS- > • : : 

3.1.1. A pre-test/pbst-test design 

A typical pre-test/post-test design includes a within-subjects cornfjwSafr%* perfbrrnance 
before and after sbme treatrnent and a between-subjects comparison of one or more 
treatment groups with a cbhtrbl group. (See Figure 7a,) This type of design is useful 
when several versions of the target test are available so that verslbris can be 
counterbalariced with test tirne. Researchers using this design to Investigate transfer 
generally try to show that the treatment grbup s performance on the target task changed 
while the cbntrbl group s did not; they would then attribute the differentia! change tb transfer 
from the source dbmalh. Figure 7b shows the predicted resui»s for a case in which 
improvement means an increase in some performance measure, such as the nurnber of 
cbrrect respbnses; the graph could be turned upside down to show the effect fbr cases In 
which imprbvement yields a decrease in the rneasure (e g:, solution time). All of the LOGO 
transfer studies described in Chapter 1 basically have this design, though sbrne (Daibey and 
blhn» 1984; 6orman and Bourne. 1983: Degelman. 1986: Brbwri arid Rbbd, 1984; Glernerits 
and Gullo. 1984; arid elements. 1985) lack a pre-test. or a control group, or both. 
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insert Flgare 7 about here 



This Study Includes a wlthlri-subjects pre-test/pbst-test cbmparlsori of performance on a 
non-cornputer debugging task. (See Figure 7c.) A mid-test was also Included to rnbhitbr 
transfer between mlhj-cburses. At each of the three test tirnes, each student took three 
types of tests (1, 2, arid 3 In Figure 7c), all of whlcfi Involved debugging a written set of 
Instructions about how to achieve a welj-speclfled goal. There were three verslbhs of each 
type of test so that the tests could be counterbalanced with test tlrne. In other words, one- 
third of the students took each version at each test time (a, b, or c In Figure 7c). The 
hypothesis was that students' ability to debug these non-cbrnputer tasks will Improve as a 
result of learning debugging In LOGO 

Several types of cdritrbl groups could have been used In this study; however, no control 
group was Included for the foiiowing reasons. One alternative hypothesis Is that students 
would Improve on the post-test purely as a result of their natural development over the time 
span of the LOGO course. The control for the effect of rti atigiDua to built Into the 
treatrhehi group since the age range of the students Is 3 years whtt^tftraRivi of the study 
Is only 6 months. In other words. If deve'bpmerital change over the ff^-montlr iitdy caused 
the Change, then the older students should be better on the pre-test than the ybuhger 
students are on either the pre-, rrild-, or post-tests. (See Figure 8a.) 

Insert Figure 8 about here 



Another alternative hypothesis is that the improvement on the post-test Is due merely to 
learning how to do that type of test. This hypothesis can be tested without a control group 
by comparing the Improvement on tests given within one session to the irnprbvernent 
between sessions. Figure 8b shows the contrasting effects. If the imprbvemerits can be 
explained simply by familiarity with the tests, then the irnprovements from one test to the 
next should te constant. However, a pure transfer effect would yield irnprbvernents only 
between sesslbris (between tests 3 and 4 and between tests 6 and 7). 

Having a control group with students in ah identical LOGO course without the debugging 
Ihstructlbn wouid be apprbpriate for showing that the ihstructlori is necessary for learning to 
occur arid that without learning no transfer occurs. Such a group was not Included Iri this 
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study because the pilot study (described in Chapter 2) already showed that students In a 
LOGO graphici ooufie did not learn effective debugging skills without Instruction; therefore, 
no transfer could be expected. Unfortunately, the transfer tests were not used In the pHbi 
study so the tie between the lack of iearning and the lack of transfer cannot be shown 
directly. However, some suggestive evidence from this study and several cdhcur^ent studies 
will be discussed In Chapter 7; 

3.1.2. A savings design 

A savings design is useful for testing transfer to target domains where multiple versions of 
the test are no« available, as In many prbblem-sblvlhg and skill acqMlsition dbrnalna. in the 
simplest case, the design involves a between-subjects comparison of two groups. One 
grbup does task A and thbn task B; the other group does the two tasks in the reverse 
order. (See Figure 9a.) Transfer Is then measured as the savings the group doing each 
task secbhd experienced, compared to the group doing that task first, as a result of 
experience with the other task. For measures such as time br the number of errbrs* 
savings wbuid be reflected by a decrease in the performance measure. For example. In 
Figure 9b, better perfbrmahce of grbup 1 than grbuja 2 on task"" 8: woold. be attributed to ? 
transfer from task A. Better performance of group 2 than group 1"' utt" iwk A would be : 
attributed tb transfer frbm tasj< B. For other perfbrrnance rneasures, such" as accuracy t^ ttw^ . 
amount accompilshed. savings actually yields an Increase in the perfbrrnance measure. Aisb, 
the actual relationship between the lines would have tb be predicted based bh the relative 
difficulty of the tasks and the expected amount of transfer: in some cases, there may be 
more asymmetry than Is depicted here. This design has hbt been used In the LOGO 
literature so far but has bean used frequently in other transfer studies; fbr Instance, Smith 
(1986) showed savings bf tbtal time fbr sblvirig tower of Hanoi isbrnbrphs, and Slngiey and 
Andersor; (1985) reported savings of total time, number bf keystrbkes, residual errbrs, and 
seconds per keystrbke for iearning a second text-editing system. 

insert Figure 9 about here 



in this study, there were two groups bf subjects. Bbth grbups received the same tOGQ 
ihstructlbn including explicit instruction In dBbugging. However, one grbup began with 
graphics and then moved intb iist-processing, while the other group took the two mini- 
courses In the reverse order. (See Figure Sc.) Performance on prbgrarnrning. debugging. 
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and ediffing (a, arid e In Figure 9c) was measured at three tlrnes daring each rnini-course 
(1. 2, and 3 for graphics and 4, S, and 6 for list-processing In Figure 9c), so. students took 
a total of 6 programming, 6 debugging, and 6 editing tests. tests were hot 
counterbalanced with test time since they corresponded to the concepts being iearned at 
that period In the course. Better perfbrrhance on graphics tests by students taking graphics 
second than by students taking graphics first can be attributed to transfer from their llst- 
prbcesslng minl-cburse. Ukewlse, better perfbrmance bn iist-prbcesslng tests by sttidents 
taking that second than by students taking It first can be attributed to transfer from their 
graphics mini-cbuise. 

Figure 10 shows the combined design. Students took one version of each type of transfer 
test at each bf three times: befbre the first minl-cburse began, between the rnlni-courses, 
and after the second mini-course ended. Performance at the three test tlrnes will be 
cbrnpared tb shbw whether the debugging skills students iearn frorn the tOGO courses 
transfer to debugging hbn-cbmputer directions. All bf the students received the sarne bOGO 
treatment including expiicit instruction in debugging; however, students took the two mlhl- 
cburses In different orders. During each mlhl-cburse, leanSnQv^ ^pgyiranlng, debugging, 
and editing slcllis was monitored at three times. Perforrmncp-'of* the:ft«r>^rbups on the 
same tests cbuid then be cornpared to show whether the SWM:. tr ansl ew d u lpagr am mini- 
course to the other and whether this transfer was equal br asyrnm^rta . . 

Insert Figure 10 about here 



The fbllbwihg sections wlil describe various parts of this design in more detail: subjects. 
Instructional methods and currrculum, data cdllectibh issues, and Lssessrneht procedures and 
materials. 

3.2 The elassrbdrh and the classes 

The LOGO mini-courses were taught at the Montessorl Center Academy In Glenshaw, 
Penhsylvahla during the i 985-1 986 schbbi year. The experimenter, who was aiready an 
experienced LbC3b teacher, served as the ihstructbr. The school had 3 cbrriputers prior tb 
the experiment (bne in each of the eiernentary classrooms and one used by the 
headmlstressj. but there was no cbmputer iristrucUbh in the curricuiurn. in fact, ?he 
computers were used only rarely, and then for prograrhmed ihstruchoh and cbrnjDuter garries. 
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M Instroction took place In a dedicated computer room. The classrbom Had two Apple lie 
cdmputeri wttH Apple LOQO \l Stadents came to cornputer classes in six groups of four 
and worked In pairs. Groups were assigned by the students' regular teacher; she was given 
only the specificatlbn that the groups should be as mixed as possible (boys and glrls^ 
younger and older students) so that any differences betwjsen the groups would bi 
attributable to the treatment rather than the age or sex distribution. Three of the groups 
took the graphics mini-course first and then the list-processing mlhl-cdurse; the other three 
groups did the reverse. Each group Had two one-hour LOGO ciasses per week for 25 
weeks. 

All of the 3rd - 6th grade children at the Mbntessori Center Acaderny participated In the 
experiment/ Twenty-two children (8 females and 14 males) ranging In age from 8;2 to 11;8 
successfully completed both LOGO mini-courses. Two of tiie original 24 children did not 
complete the course; one student left the school and one moved down to 2nd grade. They 
were replaced by two other children who rnbved up from 2nd grade; however, the dita from 
these four students are not Included In the analysis. Table 4 lists the subjects, their ages, 
their grades In school, their standardized achievement scores, whettw. they tiid a computer 
at home, and whether they took graphics or list-processing first- 6r^j9:i# link graphics first 
and then iist-processing; Group B took the mini-courses in me wmm^iM^B^ 
for each group show that Group A Is siightiy younger than Group & btit' iwformed^ ^^l^^ wr 
the standardized tests. 



Insert Table 4 about here 



3.3 Instruction techniques 

instruction consisted of 2 LOGO miril-cburses. brie to teach graphics arid one to teach list- 
processlrig. Students received 50 hours of instruction, the first rhiril-cburse was 27 hours 
and the second was 23 hours; the difference was due to the absence of introductory 
computer famiiiarlzatlon and faster progress during secbrid miril-cburse. Iri addition to 
lessons about the prograrnrnlrig language, students also received lessons designed to teach 
ihe model's debugging skills directly. These iessbris irivblved specific instruction about the 
model's goai structure, discrepancy-bug mappirtgs. arid Ibcatibri clues. 



__Tii«_ J^ontessbrl philosophy emphasizes moitl-levei classrooms: at this particular school, the 3rd-6lh grades 
were combined Into a class with brie teacher. 
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SMIii in progrifTimlhg, debugeihg, arid editing were tested at three tlrnes during each mini- 
coum. Each iubject took a total of 6 series of tests, in addltidh, each subject was gKeri 
a pre-testi a mid-test (between the two mini-courses); and a post-test to determine their 
ability to debug non-computer directidns (such as directions io set a table or distribute 
wages). 

3.3.1. LOGO skills 

Tabies 5 and 6 show the sequence of instruction and LOGO sklii tests for graphics and 
list-processing. The exact timing of the lessons varied since the second rnlnl-cburse was 
shorter; however, the order was consistent. Spaces in the secdhd rtilhl-cburse column 
Indicate lessons which were skipped or were incbrpbrated Intb bther lessons as extra time 
permitted. 

Insert table 5 about here 



The graphics mlnl-cdurse began with ah Ihtrdductlbh to InteracUMrvtM of tOGO with the 
basic cbmmands tb mbve and turn the turtie, manipulate th#^turtt#'r *pirf;v etc. Students 
were then Ihtrdduced to procedures for writing and editing prbgrfimw^ QoSi^ 5 (3 for 

the second mini-course), students learned about the REPEAT cdrtrrnSmt- and Its usefulness 
for making regular shapes and curves. In the 14th hour (11th for the second grdup). 
students were Intrdduced to local variables. During lessbri 21 (16 fbr grbup B), students 
were taught hbw tb write recursive procedures and use conditional stdp statements. 

Insert Table 6 about here 



The list-processing minl-cdurse alsd begar* with ah ihtroductlbri tb the Interactive use bf the 
basic cbfTimand, the PRil^T staternent. Students learned the distinctldri between words and 
lists and the puhctuatibri necessary fbr each. This intrbductbry material was followed by 
Instruction In writing and editing prdgrams. During lesson 3 (1 fbr the second grbup), 
students were Intrbduced tb the MAKI! command for setting global variables and td 
coridltldhai equality statements. In the 14th hbur {12th in the second rnlnl-course), students 
were Introduced to recursion, the FIRST and BUTFIRST cbmrriarids. conditional stbp 
statements, arid cburiters. During lesson 21 (16 for group A), students were taught abdut 
generating rahddm numbers and chbbsing randbrri Iterns frbrri a word or list. 
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Ail iewora were taught in a guided dlscbvery manner and ihcluded time for self-lnltlated 
prbjecti. The Intervention of the teacher In the students' work was Icept to a -minimum, but 
new commantto and Ideas were Introduced In a structured way and beginning activities for 
using them were Initiated by the teacher. Since the memory load of early programmlhg is 
high, reminders of all edrnmahds and concepts were posted on a large builetin board which 
could be vliwed easily by the students. Also, ah angle wheel was provided to aid In 
caiculating relative angles and a coordinate chart was provided to aid In position and 
distance estimation. To further decrease the difficulty of graphic? prograrnrning, Students 
were taught computation heuristics such as angle addition and the use of symmetry. 

In addition to the LOGO content in the mlhl-cburses, students were strongly encouraged to 
use subprogram structure In their programs. Subprograms were Introduced In lesson 6 (4 
for the second grbupj, before the first skill test sequence. An entire lesson on the benefits 
of using subprograms (decomposition, reusability, and cdmpartmehtallzatlbh) was presented in 
the lessbh Immediately after the first prograrnrning test. 

3.3.2. Debugging skills 

The "cognitive objective " (Greehb, 1975) of the debugging curricuiurn was to get students 
to acquire the same goai structure as the model. We hbped tb train students to look for 
and tb use cues fbr identifying, locating, and correcting bugs so that they could avbld the 
frustration of serial search. With bhiy slight rewbrdlhg of the goal structure shown in Figure 
3, particularly the interactive prompts the model gives the user, we were able tb produce a 
step-by step debugging prbcedure tb teach the students; Figure 11 shows the debugging 
procedure students were taught In terrns bf the fibw diagram of the GRAPES rnbdei to 
highlight the similarity between the model and the instructibri. Debugging skills were 
Introduced explicitly after the first debugging test (6-8 hours into the course). This timing, 
directly after students had experienced the difficulty bf debugging, insured their 
uhderstandlhg of the great usefulness of the skills being taught. After the step-by step 
debugging prbcedure was Introduced, the students used the d^boggfrrg sfepr-fo correct- thi? 
same program they had tried to debug the day before. 

Insert Figure 11 about here 



Beginning during this lesson and continuing throughout the course, the class accumulated 
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a ilat of dlscrepincy-bug mappings and awfui ioeatlon cluii. thiii dlicrepahey-bitg 
mippingi ire oquiviliht to tha knowladga in tHa propoia prbductlona and tha location clues 
are equivalent to the knowledge in the reprisent and specify prbdtictlbhs. 

During this Initial lesson, and any other time the students needed heip debugging, the 
teacher used the following 4-step sequence of approaches to prompt students developing 
knowledge and slcills: query, coaching, reflection, and recording 

the initial query approach was designed to help students to consider avallablle cues for 
Identlfylhg and locating the bug. For teaching dlscrcpancy-bug mappings, questions focused 
on the difference between the output and what was desired, what cbmmand(s) generally 
cause that type of difference, and In cases when there was more than one possibility, how 
to distihguish between them. Uhh and Fisher (1983) suggested a similar approach to 
emphasizing debugging that consisted of requiring students to propose at least two 
hypotheses about the Identity' of the bug before looking Into the program. For teaching 
structural cue use, the questions focused bh knowledge about how the program was set up 
arid where the buggy command might be In the program (neatrr^irtiiL other command?, 
before/after what bther command?, In a REPEAT statemeiil?^ or* 
Coaching was used essentially as a memory aid fbllbwihg xrm.qmf.^.^piace39\ It tnerefy 
consisted of remlridlrig the students of the clues they had Identified: • 

Reflectlori foNowirig debugging emphasized useful clues from each debugging episode. 

Students were prompted to recall what difference they were trying to fix, what type of bug 

had caused that problem, and whether similar differences were always caused by similar 

bugs. For structural cues, students were asked how the program was structured, where the 

bug had been fbuhd In that structure, arid what ciues did (or could have) helped them to 

locate the bug more easily. This prbmptlng was designed to help studerits develop abstract 

rules about the discrepancy-bug mappings and ways to distinguish them as well as about 

useful structural cues. The recordlrig approach was included to keep a written record of 

the discrepancy-bug rules for use In later debugging situatibhs. The records were kept bri 

large charts with the headings: "If this goes wrong," and "then check for this bug." 

Graphics and llst-processlhg classes kepi separate charts bri two sides of a mobile bulletin 

board. The graphics side was not visible during list-prbcessirig classes ribr the list- 

processlrig side duririg graphics classes so that the students could ribt learn the mappings 

♦ 

for a mlhi-cburse they were riot curreritly taklrig. 
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During the entire mini-cQurse, only one Hilf-hbur lesebh was devoted explicitly to 
debugging. (Trinscrlpfs of one graphics and one list-processing lesson from^ thi first and 
second mlnl-coum preientatlons are provided In Appendix iii.) During the rest of the 
course, posters with the debugging method, the discrepancy-bug mappings, and the location 
clues were available. Also, students were continuaiiy prodded to use the debugging skills 
and challenged to find hew mappings. 

3.4 Data coHection issues 

the primary goal of studying skill acquisition and transfer Is to understand the detailed 
itiechahisms and Internal structures of cognitive processes Involved; Several methods were 
used to ensure collection of data that would facilitate this understanding, the students' 
behavior oh all of the tests was videotaped to get a detailed record of the Intermediate 
steps In the solution processes. In addltlbh, students were encouraged to think aloud so 
that the goats, strategies, and knowledge Influenclhg their sblutlbhs cbuid also be recorded. 
Tb ehcburage thinking aloud, students worked In pairs for some of the tests. Also, to 
prevent students from getting stuck brt any one part bf a test, thee.axpirtm Intervened 
to provide help when Impasses were reached: Each of these mefliod||4sQt^ viHI be 

discussed In more detail In the fbllbwihg sectlbhs. 

3.4.1. Protocols 

In order to study cognitive processes, the data collected must Include rnore than 
Information about the end-product of the process. the Intermediate steps In subjects' 
Sblutlbhs reveal the pathrs) by which they reached thern. Each ">*^p in the process may 
depend on having particular knowledge abbUt the current situatibh and may produce hew 
knowledge as well. thihk-aloud protocols were used to solicit verbal expression bf the 
knowledge currently active In the subjects' shbrt-terrh rhembry (Ericsson and Simon, 1984). 

Tb capture these Intermediate steps and relevant knowledge, all of the tests were 
videotaped. Fbr ail cbmputer tests, the camera was focused oh the cbrnputer screen only: 
the videotape contained a visual record of all screen activity and ah aUdltbry recbrd of all 
verballzatlbhs by the subjects and the experirnenter. For the non-programming tests, the 
camera was focused oh the subject's paper(s) instead of the cbrnputer screen: In both 
cases, the videotape also contained a recbrd bf ah elapsed time indicator (accurate tb the 
nearest second). these recording measures should provide the informatlbh needed tb 
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aeeuntvly assan tha complax procaisei involvid In computer prbgrammlng and debugging, 
In partlctJiar, Information about tha knowiadga sabjacta had (aspaclally that wblch would ba 
cbnaldarici Input to tha dparatora In bur rnbdal) and the goals driving their sbiutlbn procaaa. 

3.4.2. Partner 

Frbiti bur past experience, we expactad that the children would have difficulty giving think- 
aioud protocols, aspaclally in such a cognltlvely demanding situatlbh as cbrnpuiar 
prbgrammlng and debugging. For this reason, chMdren worked in pairs during class and for 
some of the testing. We felt that the joint effbrt wbuld require cbmmunleatlbn of strategies 
fbr and knowledge about the task, this collaboration was used primarily for the benefit of 
the experimenter, it would have been IhterMtihg tb study the effects of joint work; however, 
the current study was not designed for that purpose, thus, one untested assumptibn In ihls 
study is that the use bf partners has hbt distorted bur results. 

There are several justifications for this assumption. First, Montessbri instructibh stresses 
coiiaboration so the students in this study were used tb working In pairs and smalj groups. 
Also, pairs were chosen such that children always worked \^\^m- psmm bf equal abHHy 
(usually also bf equal age). Research bn cbllabbratibn in prbgrammlrf^ dme^ arareau 
Webb (1984) showed that students* mastery of BASIC cbhcepts was eS^aBf^'wheh they worked 
In pairs and when they worked alone but admitted that the learning processes may have 
been quite different. Oh the bther hand, dacobsbn and dacksbn (1986) taught business 
students a course In computer programming either with small arnbuhts bf peer review br with 
ah equivalent amount bf additional instruction. They found that the students who 
participated in the peer review process had higher scbres on a content test and, rnbre 
Impbrtantly, used only 60% of the computer resources that the control group used. Hawkins 
(1983) makes similar claims fbr the pbsltlve Irnpact of cbilabbratlve work on cognitive and 
metacognitive skUls but has not demonstrated the effect. 

3.4.3. Prbddihjj 

in an attempt to minimize frustration and maximize data cbllectibh, the experimenter did 
prbvlde help when students reached an impasse or had gone far afield of effective 
procedures. For example, if students had been aiibwed tb focus on only one bug for the 
entire test time, then we would have 'oeen able tb cbilect data bhiy abbut their solution 
process fbr that one bug. intervention was desirable so that students would have an 
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opportunity to attempt ae many bugs as possible during the illbtted time, thereby maximizing 
the amount of data Mliocted. 

When using such Intervention, it is important to consider Its effects. Therefore, the 
frequency and type of Iriterventlbn was noted for all tests and will be discussed In the 
analysis sectlbn. 

3.S Procedures for assessing skiii acquisition 

Transfer Is not possible If learhlhg has hot taken place. In order to specify precisely 
which sklHs were avaliable for transfer to the non-prbgramming tasks, skill development was 
mohitbred three times during each mlhl-cburse and the savings from one minl-course to the 
other was measured and then tested for significance using two-way ANOVA (1st mlhl-cburse 
vs. 2hd mihi-cburse and graphics vs. list-prbcessing). 

Three types bf tests were used to monitor three related, but distinguishable, skills. At 
each test time, students took three tests, eacli bf which used the sarne prbgrarn goal. 
First, they wrote a program to accomplish the specified goal. Then, ^they debugged a 
purposely buggy version bf the experlrnenter's prbgrarn tb accbrnpiish fSe eame goai. 
Finaliy, they edited tlie experimenter's program. These three types of tests are represerited 
by the a, b, and c In Figure 9c. 

Each student took three series of these tests during each mihi-cburse (1. 2. arid 3 Iri 
Figure 9c). The first prbgrarnrnlrig test was taken after students had some experience with 
subprograms but before the teac!ier had stressed the usefulriess bf subprbgrarnrnirig fbr 
sirnpiifylrig the debugging process. The first debugging test was taken after special atteritlbh 
to subprograms but before debugging Iristructlbri. These first tests therefore serve as a 
baseline, that is, the level of skill prior tb explicit iristructlbri. The fbilbwirig sectlbris 
characterize each type bf test and describe the aamlnlstratlon procedures. 

3.5.1. Programming tests 

Fbr prbgrammlrig tests, the students were asked to write the program(s) to accomplish 
some particular goal. This type bf test was Iricruded tb mbriltbr develbplrig prbgrarnrnlrig 
skiiis arid tb see what debugging skliis were used spbritanebusly during prbgrarrirrilrig. Iri 
additlbri, the prbgrammlrig experience provided the students with advance understanding of 
the prdgram(s) that would be ericburitered iri the debuggirig test. This procedure decreased 
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tha comprahansion damanda of tha dabugging procasa, 1.1., tha childrah wara alraady vary 
familiar with whit tha program should do and tha ganarai way it which It should ba dona. 
Also, fimliiarlzlng tha chlldrin with tha program added ib tha cbniarit validity of tha 
debugging tests since most debugging takes place in a context where the prograrnmar Is 
alraady familiar with the program. 

Students worked in pairs for the prdgrammlng tests. They were given a prbgrar.i plan (a 
plcibrlal description in graphics or a written and oral description In llst-procas,fing) and asked 
to write the prdgram(sj. The studahts' prbgramrnlhg was vldabtapad until the program was 
complete or one class period had elapsed, whichever came first. The experimehter 
intarvened only when students were at an Impass or were going far afield of tha desired 
goal. 

The programs used in these tests were designed tb require use of all the programming 
cbricepts learned up to that point. Graphics tests all included portions which cbiild most 
afflciantly be done using a repeat statement. Later tests Included ^brtlbna which shouid 
have used variables and recursion. CBraphlcs students wrote p i uyr mill -fix- aw a farm (with 
farmhouse, slid, and four Identical cbrhstalks), a sea scene (wtth boat, son^ m^ saagulls of 
various sizes), and a garden (with two rows of flowers decrwSogt-Jn sia wn& two^ dWararvf - 
sizes bf butterflies). List-prbcessing students wrote programs ter pfay Iftadilbs wttti a user, tb 
play an Uhscrambllhg jjame with a user (using recursive stepping^^ SSoo^ a- Ml), antf to 
generate poetry (with some user input and some random selectlbrt). Ait of the prbgrams 
(graphics and llsi-prbcessing alike) cbuid have been written using multlpta subprograms. In 
general, the programs provided bppbrtuhlties for the students tb demonstrate their most 
advanced skills. The program plans and the experimenter *s correct prbgrams fbr 
accdmpllshlhg those plans are presented In Appendix IV: 

3.5.2. Debugging tests 

For debuggirig tests, the students were asked to debug a buggy version of the 
experimenter's program to accomplish the same goal as used in the prbgrammlrig test 
These tests were designed tb assess the students* acquisition of the model s skills in a 
situation where debugging was required. These tests used prbgrams that the students had 
nbt written themselves so that the bugs would be the same fbr all students. 

For each test, the students knew what the prbgram should do since they had written their 
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own version of It already. WbrRIng In peiirs, students were given the buggy prograrns online 
and askad le fix all th» bugs. The studerits' debugging was videotaped untl> the prbgrarn 
worked or on« class period had elapsed, whichever came first. 



The programs used for the debugging test were weil-struciured; in other words, they made 
appropriate use of subprograms and other LOGO substructures such as repeat and 
recursion. Six bugs were added to each program. For the graphics tests, five of the bugs 
were semantic bugs while only one was a syntax bug. Syntactic errors include misspeilings, 
inappropriate punctuation or spacing, and other errors which interrupt the running of the 
program. Semantic errors do riot stop the prograrn from running but dO cause faulty 
output. Since syntax tends to be more of a problem for iist-prbcesslng. those tests 
contained three syntax arid three sernaritic bugs. 

The bugs were choseri so that the discrepancies they caused would be fairly independent 
either In space (usually for graphics) or tirne (usually for lists). The only other criterion for 
bug ssleetibri was that there be a variety of discrepancy types in each program (size, 
orientation, location, exterit. misslngpirt, etc. for graphics and non-matching, wrong variable, 
printed variable, etc. for lists). NO attempt was rnide to control for the difficulty Of the 
programs or the bugs sirice the level of programmirig skill was developirig. Appendix V 
presents the buggy prOgrarris and the buggy output for ail six debugging tests (three 
graphics arid three lists). 

3.5.3. Editing tests 

For the edltlrig tests, the student was asked to rriake the changes rnarked on a printout 
of the program from the debuggirig test. These tests were desigried to moriltbr separately 
the deveibplrig skills for Interacting with the computer and thereby to determine how rnuch 
of programmirig and debugging time is merely due to typing and editing. They allow us to 
document whether improvemerit of debuggirig times could be due merely tb improved typing 
arid rnarieuverlng skills. That is. imprdverrierits iri grbss perforrnance may simply be due tb 
ihe Irnprovod "cierlcal" skills of rnanuevering in the editing envlrbrimerit. . 

Students worked individually on the editirig tests. The student was given the buggy 
programs bhllne and a hardcopy of the buggy prograrns with the 6 corrections marked 
clearly In red Ink. Each correction was pointed out to the student whb was then asked to 
make all the changes and run the prbgram to derrionstrate that it worked properly, if the 
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prof rim did not work, either because one or more of the changes had not been made or 
becaase the stodim had introduced new bugs, the student was asked to correct the errors 
and re-nin the program until It worked properly. The entire editing test was videotaped. 

The programs used for the editing tests were the same programs used in the debugging 
tests. Students were already farnlllar with the prograrn and rnay have even rernernbered 
some of the ^ugs they he ' found on the debugging test. The marked hard-copies are 
presented In Appendix VI : 

3.6 Procedures for assessing debugging transfer 

Transfer is "simply applying Information about a I<n6wn category to a new Instance" 
(Bassok and Holyoak, 1986). the goal of the transfer assessments Is to discover which of 
the knowledge and skills available for transfer are actually applied In new instarices. the 
new Instances must, therefore, be recognizable as members of the sarne category and the 
skills must Indeed be applicable. 

Choosing an appropriate "far" transfer task rarely has a p rtn cp i tf r aais. Typically, 
researchers have a plausible but vague notion of the similarity betweerr the source arid 
target tasks (e.g.. McGllly et al., 1984). The debugging model, described In Chapter 2. 
provides a way to advance the specificity of predicted transfer effects arid the choice of 
tasks for debugging skills. 

Transfer can only be successful when the new ddmairi is viewed as ari Iristarice similar lo 
the base ddmairi iri a way that would mal<e use of already learned skills appropriate, the 
choice of transfer tasks must be based bri the slmllanty of the required skills; For this 
reason, the transfer test used In this study was designed to be similar to LOGO debuggirig 
in terms of the debuggirig situation, the types of information available, and the location cues 
available. The tests were designed only to test the trarisfer of debugging skills (not 
progrsmmlng or editing slcllls) since debugging skills were the subject of bur task analysis 
and explicit IristrUctlori. 
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3:6.1. Simliarity between debugging programs and debugging directions 

Three types of transfer tests were designed, all of which Involved detection ind correction 
of errors In a wrKten set of instructions about how to achieve a well-speclfled goal. This 
task is similar to bOGiS debugging in three ways. 

First, before the transfer tests begaht the teacher gave Ihstructlbhs which were designed 
to highiight the debugging nature of the tasks; Program debugging Is viewed as a situatloh 
where a programmer has giveh the computer cbmmahds, the computer foiiows the 
commands perfectly, bat something goes wrong because one of . the commands Is wrong. 
The debugger's job Is to find the bug In the cbmmarids and fix It so that next time it will 
run correctly. The Instructions for debugging directions mimic the program debugging 
sltuatibh: 

"Today I would like you to read three Jtbrles^ in each story, someone qlves 
sbmebrie else directions. The person foiiows the directions perfectjy, but something 
jgbes wrbhjj because one of the directlor^s is wrong. Youj" job Is to find the 
problem with the directions and fix It so that next time It will be done correctly," 

The cover story for each item reiterated these Instructions. 

•»* 

Second, Information about the desired and actual output Is available In debugging 
situatlbhs even before the programs are viewed. For debugging directions, similar 
information was provided before the directions could be viewed, in two of the three test 
types, this Ihfbrmatibn was in the form of pictures; however. In the third. It was In the form 
of tables. From the pictures and tables, subjects couid have gathered clues about the 
identity of the bug and the probable iocatlon of the bu§ just as they could In the program 
debugging situation. 

Third, the iists of directions were structured in ways sirhiiar to LOGO, primiairlly like 
subprbjjrams but also like repeat statements. This was accomplished by the addition of 
headings between sections of directions to label their purpose. Headings were printed flush 
with the left margin, whereas the directions relating to them were indented. Subjects could 
use the headings to determine which sections of the directions were likely to contain the 
bug just as they could use the subprogram names to guide their search for program bugs. 
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3.6.2. Possible Solution strategies 

The following example wHl show how the rnodel's brute force strategy (low Ihforrhatlbh 
search) and selective seirch strategy (hlg^ Infbrrnatlon search) would solve one of the 
transfer items. Figure 12 shows the plan and butcbme for the furniture arranging problern. 
Table 7 lists the accompanying directions. Before viewing the figure, students read the 
fbllbwihg cover story. 

Mrs. Fisher was rhbvlrig Into a hew house with the help of two movei^. She 
asked them to arrange the furhlture in her house and gave them a iist of 
directions to follbw. The movers fbllbwed the Instructions perfectiy . but there was 
one problem with the directions so the furniture was not arranged correctiy. 

the next page shows the way Mrs. Fisher wanted the furhlture to look and the 
way It looked after the _mpvers arranged It. _ Use these pictures to help yoo find 
the problem with Mrs. Fisher's directions. Then fix the directions so the rnovers 
could arrange the fUrhltu'^e correctly. 



Insert Figure 12 about here 



Cbmparlsbh of the two floor plans reveals that there Is a table out of:,i)lace. Closer 
Inspection may reveal that the table Is In the llvlhja room. One rnlght also notice that the. 
table has been placed between two chairs In both drawings arid hypothesize that the^ 
confusion resulted from a misunderstaridlhg of which two chairs. All of the directions begin 
with "Here are the directions so-and-so gave to sb-arid-sb." arid erid with "©hange or add 
brie thirig tb fix sb-ahd-sb*s directions." In most cases, the directions are divided Iritb three 
parts; here, one part describes how tb arrarige the diriihg rbbrn, one part the living room, 
and one part the kitchen. 

Insert Table 7 about here 



Solving this problem would be quite tedibus fbr sbmebrie using a brute force strategy, as 
many children did. The solver would read each line and check the picture tb make sure It 
was correct Uritll the Iricbrrect directibh was located; A soiver who knows to look for a 
misplaced table might scan the directions Uritil reachlrig brie describing the placernent of a 
table. This would lead to false alarms on the lines describing the three bther tables iri the 
home (especially the two which are described prior to the correct table). A solver who 
knows to look in the directions fbr the livlrig rbbm will Ignore the dining room directions and 
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. focus only on tHe IMng room ones. Depending on the other available information, the solver 
rtiight chectc each of the living robrti cortirtiahda or only the onea referring to tabiea: A 
solver wtio noticed that the table was between two chairs could scan for a phrase about a 
table between cHalrs. Since the kitchen directions foiiow the incorrect direction and are 
independent of it, they would be Ignored by all solvers who iocate the bug on their fii^t 
pass through the directions. 

Solvers with ail of these strategies cbuid Ideate the bug and add the inforrnatlon to define 
which two chairs the coffee table should be between. The search process, hot the success 
rate, is what should distinguish the different strategies. However, solvers who search more 
of the directions might be more likely to false aiirrti and therefore be less successfui. The 
remaining eight tests are presented in Appendix Vii: 



3.6.3. Transfer tests 

The three types of tests were chosen, on the basis of inforrnai pretests, to get a range of 
difficulty. Three items of each type were constructed, one each for the pre-, mid-, and 
post-tests. Ah equal nurnber of subjects were given each Item at each test time. The 
easiest problems Involved directions for arranging sbrriethirig (simifTg a table, building with 
blocks, or arranging furniture). The next easiest problems Invof^^ aiw cti dn s for distributing 
somethlhg (paying wages, delivering trees, or ordering food). TR^ most- difficult problems 
Involved directions for traveling somewhere (playing golf, visiting airports, or running errands). 
The tasks were always presented In order of Increasing difficulty so that students would not 
do poorly on an easier test purely as a result of being frustrated by a harder one. 

The transfer tests were given In the cbrnpater classrobrn at the teacher's table. Students 
worked on the tests individually and were asked to "read and think ^Ibud" while they 
worked. All work was done In a test packet which contained the three stories, pictures or 
tables cbmparlhg the desired outcome with the actual butcbrne, and a list of directions, 
the entire transfer test was videotaped. In this case, the camera was fbcused bri the test 
packet as It lay bn the teacher *s table; the teacher made sure the student kept the packet 
flat while working. 

The videbtaped recbrds bf the students' performance were used to determine their search 
strategies In addltibri to the accuracy and timing bf their answers. The significance of pre-, 
mid-, and post-test differences In qualitative strategy classificatibhs will be tested using a 
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anaij^ls, and the significance of differences In quanltatlve analysis will be tested using a 
two-way ANOVA (test time by test type). ^ 

To summarize, the experiment is a combination pre-test/post-test and savings desigh In 
order to assess the transfer of debugging skills (and support skills) from one LOGO context 
to another and to assess the transfer of debugging skills frbrn the tOGQ contexts to non- 
programming contexts. The debugging model, described In Chapter Z was used extensively 
for designing the Instructlbh and the transfer tests so that learning and transfer would be 
maximized. Chapter 4 describes the savings from the first mlhi-cburse to the second and 
Chapter 5 discusses the transfer of debugging sldlls to non-programming domains. 
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4. Skin acquisition 

tha savlnoa saan on tha sacond mlni^oursa as a result of having taken the first reflects 
the amount of trahaferable skill the students learned. Two clear, but relatively uninteresting, 
savings have already been mentioned: the students did hot need to relearh how to use the 
computer and they progressed more rapidly through the lesson sequence so the second 
mlhl^:burse took brily 23 Instead of 27 hours to cbrnplete (85% of the first rnlnj-course time). 

Students were expected to recognize the second LOGO mini-course as a new ihstanca 
where previously learned prbgrammihg. debugging, and editing skills could be applied. To 
the extent that tha appjicatlon is appropriate, the transfer should be pbsitlva. Transfer 
would be negative In a case where appilcatibn of previously learned skills was detrimental in 
the new situation. 

Many skills would be expected to transfer frorri the students' first rnlnl-course to their 
second. Most bbvibusiy, the second mini-course took less time because the ihtroductbry 
computer famliiarlzatibri was hot necessary (learrilrig to insert the disk, turn on the computer, 
rah the printer, etc.) Also, graphics and iist-prdcessihg use the same editor so the editing 
commands are Identical. F>r6cedures are ran in the same manner, except that students 
must type CS (clearscreeh) before each graphics run to reset the screen frorn the last one: 
Several programming concepts might also be expected to transfer, such as the use of 
subprograms, variables, arid cbriditibrial staternerits; Sbrne radirnentary syntactic skills would 
be expected to transfer: the inportance of correct spelling, spaces, arid perhaps the use of 
a cblbri before a variable, bat beybnd that, iist-prccesslng has many syntactic rules which do 
not occur in graphics and little transfer would be expected. 

Most impbrtaritly. rriuch bf the debagging skiii students were taught should be transferable 
to the second mihi-cburse. Certalrily, the goal structure wbuid be Identical, as would most 
bf the iocatibn clues. The discrepancy-bug mappings would differ, except for sbriie bf the 
syntax errors, but the prbcedure fbr using thern would be the same. 

4.1 Debugging skills 

The goal bf the debuggirig analysis was to document which of the rfibders skills the 
students were able to acquire from the direct Instructibri provided In both 10(30 mini- 
cburses: There was no cdmparlsori group (a group that got no explicit instructibri iri 
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dobuggino); however, the results from this analysis can be compared to the results from the 
pilot study (Cirver ihd Klihr, 1986) described In Chapter 2. This section wiil show that 
when given debugging instruction based on the performance model, students were able to 
acquire effective debugging skilis. Without such instruction, students In the pilot study 
debugged podhy. 

Debugging episodes were transcribed directiy in terms of the model's goal structure, 
frahscrlptidh sheets were copies of the rhbders goal structure with spaces where the goal 
names used to be. Episodes were divided into cycies based on the teat goal. A new 
cycle began each time the subjects rah a program or ran a series of programs without 
doing anything else In between; these program runs were recorded In the TEST space. 
Comments about whether or not the outcome of a test matched the plan were noted In the 
EVALUATE space. Similarly, comments about the discrepancy and the proposed bug were 
written in the DESCIRiBE and PROPOSE spaces; comments about the knowledge of the 
program structure and clues for where th^ bug might be In that structure were recorded in 
the REPIRESEI^JT and SPECIFY spaces. The programs students edited In their search for 
the bug were noted In the FIND space. Commands they read were entered In the 
ii^TERF»RET space and their assessment of the commands' cbrrecthess was entered In ttie 
CHECK space. Finally, any comments about the change that needed to be made were 
entered In the CHANGE space, while the actual replacement was entered in the FjEPj-ACE 
space. The order of comments and actions was preserved by numbering each entry. The 
time at the start of each cycle was aisb entered on the transcript. In addition, all 
experimenter Ihteractibh with the pair of students was recorded on the side of the sheet, 
labeled with an E. and numbered In sequence with the other events. 

Figure 13 shows ah idealized trahscrlptibri for the last part of the l^bETRY test. The 
example is "idealized'' because subjects had difficulty contihuing to give prbtbcbis so they 
rarely made comments about all of the goals in any one cycle: comments from a variety of 
subjects have been combined to give a cbmpiete example. e example transcript begins 
with a test of the program I^OETRY. A negative evaiuatlon is Indicated by the cbmrhent. 
"Oh rior The discrepancy was described as. "it used the wrong name" and the bug 
proposed as, "It has the variables mixed up." One student asks, "which subprogram should 
we try?" Both know that the program representatidh includes subprograms. The other 
specifies the buggy subprogram as "goodbye." They edit the subprogram GOODBYE and 
scan for the variable. :NAME1 is Isolated and understood to be the wrong variable name. 
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'It SHould bi hilld/ siyi one student. The students then replace :NAME1 with :HEbtO 
and exit the editor to retest the program. A trace of the model using the same knowledge 
to debiig the same program is one of the example traces provided in Appendix il. 

Insert Figure 13 about here 



From the debugging transcripts, several related aspects of debugging sKill were coded to 
probe the students' acquisition of different parts of the rnodel's skills: achjevernent, speed, 
efficiency, ciue gathering, and search strategies. 

4.1.1. Achievement 

One Indication of debugging siclij is the nurnber of bugs students were able to fix during 
the allotted hour. Most of the results of the skill analyses are presented in a fashion 
sirnilar to Figure 14 so it will be described In detail. Figure 14a shows the performance of 
groups A and B on the three graphics tests. Group A took the graphics rninl-course- first 
and group B took It second, after having had list-processing experience. Figure 14b shows 
the performance of groups A and B on the three jlst-processing tests. Here, group B took 
the mini-course first and group A second. The irregular pattern from tOTt'. 1 to 2 to 3 oh 
these graphs reflects the differences In problem difficulty. TTm tems ^ fim hot 
cbUhterbalahced since the jsrbgrarhrhihg concepts used oh the feels reflecM the order of 
instruction In the course. 

There are several more impbrtaht patterns to extract frbrn these figures. The first pattern 
to consider is the relationship between the graphics and list-processing graphs. The second 
pattern to extract from these graphs is the relationship between the lines reprojienting the 
two mihI-coUrses. A consistent pattern between the first and second rnihi-course lines oh 
each graph would Indicate the savings resulting from the first mihi-cburse experience. 
Figure 14c sumrharizes the results; it highlights the two Important patterns (graphics versus 
list-processing and first versus second rhihi-cburse) and deemjDhasizes the Irregular pattern of 
the three tests within a mini-course. For the purpose of this graph, ihe within mini-course 
tests are cbhsldered essehtially as three trials so the scores have been averaged to yat a 
score for each rhihi-cburse as a whble. For the rerhaihing figures of this type the 
discussion wili focus only on these patterns of results. 

Figures 14a ahd b show that durihg the second rhihi-cburses. all of the students fixed all 
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of tho tlx bugs within the hour, wheriis. In the first mlnlH:burseSi they did not, F (1,76) - 
23.15, p < .01. Figure i4c shows the aggregate results; the improvement from the first 
mihI-cbUrse to the second minl-cburse was the same for the graphics and Hst-processing 
tests. F (2,76) - .06. 

Insert Figure 14 about here 



4 1.2. Speed 

The debugging speed was also measured, in cases when the students corrected all six 
bugs, this Included ail debugging time up to but not jnciuding the final run (when the 
program worked correctly). When the students did not correct all six bugs, the time was 
measured up to but not jnciuding the program run that confirmed their last correct fix. this 
adjustment excludes time at the end of the session spent on bugs ihni were never 
corrected; thus the speed measure inciudes oniy time spent on bugs that were fixed. For 
all pairs, the total time was divided by the number of bugs fixed, the model makes no 
predictions about the absolute debugging time, but the time would be expected to decrease 
as debugging skill Improves since strategies would shift from teote^lorce to more focused 
search which requires fewer subgdals. 

__ _ __ _ _ ^ -^-. S'l. , 

Figures 15a arid b compare the performarice of students on thy grapWc g and iist- 

processlng tests in the first and second rttHii-cdUrses. the graphfcs groups took almost two 

miriutes Ibriger per bug than the list-processing groups. F (1,76) = 7.45. p < .01. Figure 

ISc shows the same results iri terriis of the tline savings bri the second rfilril-course tests as 

a result of having taken the first mini-ccurse. Savings for the list-processirig iests was about 

half arid for the graphics tests was about one-third. F (1-76) = 42.69. p < .01. 

Insert Figure 15 about here 



4,1,3, Efficiency 

As debugging skill iriiproves, students should also take fewer cycles (each isolated test 
goal initiates a cycle) to fix each bug. Pirfect debugging (wheri clues are available arid 
accurate) requires only one cycle to locate and correct each bug. To assess this 
Improvemerit, the riumber of cycles per bug was measured. As with time, the total number 
of cycles was used In cases where students corrected ail six bugs: hdwever. the riumber of 
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cycles up to the last correct fix was used In cases where all the bugs were not corrected. 
Once agalhi the number of cycles w&s divided by the number of bugs fixed. 

Figures iSa and b show the Improvements In efficiency from the first mihi-cburse to the 
second for each test. F (1.76) - 12.64, p < .01. The graphics groups took more cycles 
than the Hst-processlng groups to fix each bug, F (i,76) - 62.46. p < .01. In the second 
mini-course, the Hst-processlng group was averaging close to the perfect one cycle per bug. 
Several groups of students actually tbbl( fewer than one cycle per bug because they fixed 
several bugs In one program without exiting to retest the program In between. Figure I6c 
demonstrates the savings for each course as a whole. For both groups, the savings was 
approximately 1 cycle per bug. 

Insert Figure 16 about here 



In order to account for the differences between the two groups and for the savings from 
the first miril-cburse to the second, the debugging process was considered In inbre detiH. 
The following descrlptlbhs will cbhcehtrate more on the savings ^M^KOte Uto mini-courses 
rather tlian on differences between tests within a mlh^ rjourse. " . 

4.1.4. Pre-search clue gathering " • v. 

Comments about the discrepancy, the bug, and the bug's Ibcatidh were scored as correct 
or Incorrect and as made prior to or after the first command Identified as the bug. As 
students' i<nowiedge of discrepancy-bug mappings and of Ibcatlbri cues Increased, the 
propbrtlbh of correct comments should Increase. The total number would actually decrease 
as they they improve since debugging will take fewer cycles. Also, the proportion of 
comments made before suggesting a command as the bug should increase if the students 
learn the goal structure of the model which stresses the value of seeking cues to narrow 
the search. 

Students made very few comments overall, perhaps because the task was so cognitlvely 
demanding. Stuc^nts described the discrepancy aloud for about half of the bugs, but thty 
proposed bugs for only about 1/4 or 1/3 of the bugs prior to begiririlrig their search. 
Location descriptions were more freqtr^r{ than bug proposals but were still offered for less 
than half of the bugs. Students in the second mini-course made fewer comments about the 
discrepancy, the bug, and its location because they took fewer cycles to isolate each bug. 
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Figure 17 shows the percentage of these commenis that were accurate for aj dlscripihcy^ 
b) bug, and c) Ibcatlbh comments; Accuracy was slightly higher for iist-proces9lng 
debugging. Comments about the identity of the bug were least likely to be accurate. The 
savings between mini-courses was minimaj. 

Insert Figure 17 about here 



However, Figure 18 shows that the savings between the first and second mlnl-cburses Is In 
the percentage of- comrnents made prior to beginning to search for the bug. Partlcularjy for 
bug proposals, the students are learning the value of pfdpbslhg the bug before searching so 
that the search can be selective. 



Insert Figure 18 about here 



4.1.5. Faulty search 

At the same tlme» evidence of poor debugging behavior wbui& tai expected to decrease. 
Such behavior can be measured as the number of 4MMer^liibjvdgfams the students 
errohebusly edit, the number bf false alarms ^hey make (cbri^^t>dMminte^Meritlfied as the 
bug), the number of times they abandbn their search fbr a partictifir^i'iiu^ the amount 
of help they need from the experimenter. 

The number of times the students Ibbked Into a subprogram that did hbt cbhtalh the bug 
(or infbrmatlbn relevant to the bug such as variable values) should decrease as the students 
learn tb use dues fbr iocatfrig the bug. The subprogram structure of the buggy programs 
was easy for the students to recognize because the subprograms that had been Ibaded 
were displayed on the computer screen at the beginning of the test. Students rarely 
misjudged which subprbgrani did which part bf the prbgrarri because the subprogram names 
were related to their function. The mean number of times subjects ibbi<ed ihtb a prbgram 
that did hbt cbhtalh the bug ranged from 2 to 3 per test (I.e:. per 6 bugs). Most of these 
errors resulted from forgetting the names bf the prbgrams or fbrgettlng what subprograms 
existed. The amount of brute force search {reading and checking each command in a 
program) did decrease from the first mini-course to the second. The number of instances 
bf brute force decreased from 28 tb 14 for the graphics tests and from 13 to 1 for the llst- 
processlng tests. 
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Miinjr students actaally focased too mach on the sabprograms, forgetting that they were 
called by another program. For example, most students added a SETPOS command to the 
WAVES subprogram when the waves were positioned incorrectly rather than changing the 
faulty SETP@S command In the main SEASHORE program; they never looked Into the 
SEASHORE program to see whether a pbsltlbhihg attempt had been made. Also, when 
testing subprograms independently on graphics tests, students were bothered when a figure 
ran off the screen even If It had not done so when the main program ran; they often added 
pbsltlbhihg cbmmahds tb the subprogram even when there was hb pbsltrbhing bug. 

The number of correct commands which were mis-idehilfled as the bug (false alarrns) we;re 
taliied. The false alarms were also categorized according to whether the students actually 
changed the command or only proposed the change and whether the command was in the 
same subprbgram as the actual bug. Improved search strategies shbUld decrease the 
number of false alarms, particulariy those In subprograms other* than the one actually 
cbhtalhlhg the bug. In fact, the number bf false alarms dropped frbm 206 to 150 from the 
first graphics mini-course to the second and even more dramaticaU|i from 87 to 27 for the 
list-processing ^ests. The difference between the magnitude ol;:fih^:alewia for the two 
different LOGO domains Is not surprising In view of the vast differifidr Jn- iH^tbtal number 
of commands In the programs and the greater slmliarlty hntwrtrrrr' In s tMr ii i ot each 
cbmniahd (e.g., there are many ihdistihgulshable FD commands In one gr aphics program). 
The percentage of false alarms In the wrong subprogram remained at 15% fbf bbth graphics 
mini-courses but decreased from 31% to 7% from the first to the second list-processing 
mihi-cbUrse. 

The high number of false alarms accounts for many of the extra cycles the students toolc 
tb correct the bugs. Because punctuatibh is sb cbrhpllcated In !ist-processlhg, poorer 
students adopted a trial and error replacemont strategy for fixing punctuation (brackets to 
quotes, quotes to bracicets etc.) when they could not identify the bug; 

The number of times students abandon their search (which the model would never do) 
should also decrease. In fact, subjects rarely abandoned their search (though this is partly 
a result bf the availability bf help from the experimenter). There were 13 instances bf 
abandoned search during the first graphics mini-course and 10 during the first list-processing 
mihi-cburse. In the second mini-course, the number of abandoned searches decreased to 9 
and 4, In additlbh, students were mbre likely to restart ah abandoned search in the second 
mini-course. 
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4.1.6. Incorrect changea 

For each bug that waa correctly Identified, the hurnber of changes it took id gat the 
correct coffimand waa tallied, incorrect fixes were scored as either syntactic or semantic 
errors, incorrect fixea should decrease as the students becorne better prograrnrners, though 
not necessarily aa they becorne better debuggers. 

Attempts to determine the appropriate argument for a command once the bug had been 
Identified accounted for ah average of 1 extra cycle per bug. There was ho Irnprovemeht In 
the subjects' ability to make changes once the bug had been Identified on graphics tests: 
The debugging Inatructldn did not provide any help for generating the correct cbmrnahd 
once the bug had been identified. For list-processing, however, the Identification of the bug 
alrnbst specifies the appropriate change. For exarnpie, discovering that a variable has been 
printed Inside a Hit Implies^ that the appropriate fix Is to remove It from the list, whereas 
knowing that a particuiar ieft turn was not enough does not irnpiy what the turn shouid have 
been, only that It should be more. Extra cycles to determine the correct fix decreased frorn 
an average of .4 per bug to .1 per bug as students became more familiar with the 
appropriate changes for each bug. 

4:1.7. Experimenter help 

Each Instance of experimenter help was scored for the type of ihfbrmatlbh It provided the 
students: description of the discrepancy. Interpretation of a command, recognition of a false 
alarm, identiflcatibn of the bug, ideation of the bug. specification of the change* 
recdrnmehdatlbh of strategies, or reminder of something they had previously done or said. 
As students begin to debug iike the rnodei. they should need less heip frorn the 
experimenter. In particular, as debuijgihg strategies Improve, students should heed less help 
gathering clues to narrow their search, less help correcting false alarms (since they shouid 
be rnaking fewer), less help choosing strategies, etc: For the list-processing tests, the 
amount of help subjects heeded decreased markedly frbrri 177 instances In the first mini- 
course to only 80 in the second, the biggest difference for the iist-processing group was 
on the second test which included FiRST-BOTFiRST recursion, cbnditionai staternehts. and 
counters. Apparently, the group that had graphics background had less difficulty dealing 
with these concepts than the group with no prior experience, perhaps because they had 
encountered tail recursion and conditionai statements before However, the amount of heip 
heeded oh the graphics tests did not decrease: there were 150 instances of help in the first 
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mini-course and 164 In the second. Much of this heip is reiated to other poor debugging 
strategiei such as the high number of false alarms and Incbrrect fixes. 



4.1:8: Additional testing strategies 

Finally, there are many addltlonaj strategies which could facllltati debugging, such as 
doing extra test runs to better describe the discrepancy and specify the location of the bug. 
testing subprbgrams rather than the main program when appropriate, using moclc Input 
during test runs rather than taking the time to use correct Input, making multiple changes 
between test runs when bugs are reiativeiy independent, seeking additional information from 
the program titles or from within subprbgrams hbt cbhtalhlhg the bug (such as Ibbkihg fbr a 
variabie setting), or seelcing information from the mappings chart: in fact, students used 
these strategies bhiy in a few select Instances when they were recbmmehded by the 
experimenter: Many of the students did. however, begin testing subprbgrams separately 
after being shown how useful a technique It was. Since these strategies were not part of 
the explicit Instruction (because they are hbt a part bf the mbdei). It Is hbt surprising that 
students did not use them well. 

4.1.9. Reaction to debugging ihstrucilbh . 

Students were eager to learn better debugging strategies wheir itaf :t csnw into ciass the 
period foiiowlng the first debugging test. Debugging the first test prbgrartr ivw <tl^^ for 
students In both the graphics and iist-processing groups: Many of the students never found 
all six bugs within the hour test period. The entire debugging iessbn, inciuding dei^ugginp 
the first test program using the new strategies, took only half an hour. Mbs^: of the 
students made cbmmehis like, "Why didn't you teii us this before?" They had exper ^nced 
the frusiratibn bf brute fbrCQ debugging and were receptive to more successful methods 

^fter the one (and only) iessbn focusing on debugging, students used the new stramqiPS 
frequently, especially asking themselves which subprogram was likely cbhtalh a pr.'icc. ^.r 
bug. They did not, however, make use of the list of discrepancy-bug mappihtis 
frequently. Some of the more common mappings were memorized early, but many stud'^-.^ ^j 
used the reference chart brily as a last resbrt. Nbhetheless. their debugging skills we*'^ 
Impressive by the end of the course. (They did at least as well if hbt better than the 
LOGO teachers tested by deni<jns (1986) on the same programs.) 
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4.2 Programming skills 

The gbil of the programmlhg skijj ahaiysii was to describe the level of prdgraitimlhg ability 
stadente acqalre, partjcuiarly ekine related to debugging skili. The analysis wili emphasize 
the amount of structure students build Into their programs since proper structuring makes 
debugging easier (Korson and Vaishnavi, 1986). Debugging strategies will also be 
emphasized so that the spontaneous strategies used on the programming tests can be 
compared with the strategies used on the debugging tests. 

The data for the programming analysis inciuded the students' programs, the output they 
produced, and the videotapes of their sdlutloh process. From the videotape, four classes of 
programming behavior were transcribed: code writing, code changing (without having tested 
the prbgrarh), code testing, and code debugging. In addition, the students' comments about 
how they were structuring their program(s) and any comments made by the Instructor were 
transcribed. Time was not measured because students rarely cbmpieted the programs within 
the one hour limit, especially in the graphics mlnl-cdurse. and because so much 
experimenter intervehtibn was required that the time measure would be difficuit to interpret. 

4.2.1. Achlevemem \ ' ' 

One measure of the students' programmlhg ability Is how much of the goal students 
accomplished during the allotted hour. Each program goal was divided Into units according 
to the foilowing principles: For graphics programs, each major part of the picture counted 
as one Unit. For example, the barn In the farm program had two units: the base and the 
roof, in addition, independent grouping of units (such as putting four corn stalks together) 
and pbsitibning bf units (such as placing the sun in the upper right hand cbrner of the 
screen) counted as Units themselves. For list-prbcessihg programs, each query (asking a 
question and taking the user's input), conditional statement, response, and global variable 
setting counted as a urilt. In addltlbh, several types bf user friehdiy units were scbred, 
such as greeting the user, prbvidlhg instructibhs. thanking the user, includihg WAIT 
statements, printing blank lines, and addressing the user by name. Driver programs counted 
u3 units for both graphics and llst-prbcessjhg. 

Each of the programs written by the subjects was then examined to see which of the 
units had been completed, which had been attempted: and which had not been attempted. 
The achievement measure for each pair was the percentage bf units they cbmpieted. As 
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programming skill Improves* subjects should be able to complete a higher percentage of 
unite; this section Will provide the qualitative results along with a general cbaracterization of 
their meinlng. Appendix VIII cbhtalhs the detailed description, Including the total nurnber of 
units tor each of the experimenter's programs along with the number of student programs 
containing that unit. 

Figures I9a and b show that students accomplished significantly more of the program goal 
on the iist-prdcessing tests than on the graphics tests, F (1.68) - 58.49, p < 01. 
However, students in the second mint-course accortipiished only slightly more than students 
in the fK' i course, F (1,68) - 8.32, p < .01. 

insert Figure 19 about here 

Sraphi^jS ..iudents b;3£j^n with straight-line figures and/or avoided figures with curves. This 
trend is evident since students tended to attempt and avoid the same units. On the FARM 
test, most students attempted the barn and sHo and avoided the corn stalks. 6.. (he 
SEASHORE test, most students tried the boat and sali, and few tried the seaguiis. Most 
students started with the flower in the GARDEN test and never had^llttie' for the butterfly. 

For the llst-prbcesslhg tests, the students usually began wjth the fuhdarnentai units of the 
program and added the user friendly units later (if time permitted). For the MADLIB test, 
students began with asking the user for the necessary words and printing the story; for the 
SCRAMBLES test, students began with setting the global variables for the probierns and 
then wrote the recursive function to ask the questions: and for the POETRY test, students 
started by asking the user for the necessary words, setting the global variables for the 
rhymes, and printing the poem. Many pairs wrote the base part of the program during the 
hour, but few students got far enough to add the user friendly units. 

4.2.2. Structured prbgrammihg 

Another measure of programming ability is the amount of structured programming. For 
LOGO programs, structured programming can be defined as appropriate use of line breal<s. 
subprograms, repeat statements, variables, and recursive calls. Using line breaks effectively 
Is the most rudimentary structure a LOGO program can have. LOGO can handle lines of 
Up to 256 characters so most programs could be typed without ever hitting < return >: 
Alternatively, commands heed hot be kept together so programs could be entered with a 
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< return > after each Ihciepehcieht ebrrimahd (I.e., those that are hot argumehts to bther 
commands). Neither of these strategies groups commands according to what part of the 
program plan they accbmpiish; such a strategy would be useful fbr later debugging and 
mbdifyihg. For example^ many graphics programmers keep bh the same line all commands 
executed after a PenUp command but before the associated Penbown command; this allows 
easy distlnctlbrr between lines of commands that draw something and lines of cbmmands 
that bhiy move the turtle. Using subprograms, repeat statements, and recursion also serve 
to organize programs by grouping commands. In addition, using variables groups WkB 
arguments as well as allowing fbr similar fuhctibhs to be accomplished by one program with 
different inputs. 

Once again, the experimenter's program was used as a standard. One point was given 
for the use of appropriate line breai<s, one point for every subprogram call, one point for 
every unique repeat statement (multiple uses of the sarhe repeat statement were not scored 
because they should have been embedded in a repeat statement themseives or written as a 
separate subprogram), one point fbr each variable used, and one point fbr each recursive 
call. Each of the subjects* solutions was scored for structure in the same way. For 
comparison, the mean scores are reported in Figure 20 as percentages of tbe total for the 
experimenter's program. The standard scores are presented in Appendix Vlil. "-v 

insert Figure 20 about here 



Students used more program structure In the list-prbcessing course than In the graphics 
course (F (l,68j - 92.04. p <.01), perhaps because some of the programs required 
structure In order to function properly (e;g., recursing through a list requires setting the list 
In a separate subprbgrarh). Oh the other hand, graphics figures can almost always be 
drawn in line-by-line fashion (however tedious this may be). For example, one student in the 
first graphics mlhl-cburse created one corn stali< by cbmblhlhg FDs and turns. She did use 
a repeat to make a curve, but she retyped the repeat statement six times (twice for each of 
three leaves) just to make one corn stalk. Then she systematically retyped all of the code 
fbr the one corn stalk to make a second one. She would have continued In this fashion if 
the instructor had not suggested that she move on tb the silo and barn. In cases where 
students did use subprograms, they rarely wrote driver programs to combine them. \A/hen 
such programs were written, they used chain calls (A calls B. B calls and 6 calls D 
rather than A calling B, C, and D). Students did hot improve from the first mini-course tb 
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the seconds F \^,B8) » 2.49. Even stuclents In group B who had used much program 
structure iff the llit*proceiilng mini-course used very little after switching to graphics. 

4.2.3: Commoii errors and misconceptions 

the students misconceptions and commdn bugs were also catalogued In order to 
characterize developing programming skili. Error^ and misconceptions were consistent across 
mlnl-cburses; In some cases, students In the second nilhl-cburse actually made more errors 
or demonstrated more misconceptions because they accomplished slightly more and used 
slightly more advanced structuring. 

the most frequent errors in graphics programming were inaccurate arguments (moving, 
turning, or repeating too much of tod little^ Students also made dlfectlbh errors by turning 
left instead of right or vice versa: Occaslonaiiy, they forgot to type part of a command, 
typed part of a cbrrimahd twice, put spaces where ihey did hot belong, or omitted spaces 
Where they did belong. 

The most common mlscbhceptlbhs In graphics related to moves between figures and 
drawing figures wltli curves. Students frequently forgot to matois'^noiv^^mesn figures, 
especially when using repeat to get several idehticai figures rsu(^ imr mrn stalks) so 
the figures were arawn on top of eeich other. When they did rerrterMser'thV'' move, they 
often forgot either to pick up the pen, to put down the pen, or boSn PO and PD 
commands were also frequently misplaced (after the move, for instahce). Students also had 
difflcuity remembering how to use a repeat statement to draw a curve, they frequently 
confused the two rules for curves: the FD distance times the repeat number equais the size 
of the curve while the turn times the repeat number equal3 the angle of the curve. Also, 
students had difflcuity combining cun/es: they usually failed to make the appropriate turn 
between curve segrnehts. 

the most common errors on list-processing tests were punctuation and spacing errors. 
The most frequent of these was forgetting the parentheses around a print statement when 
two or more elements were to be printed. Students had difficulty uhderstahdlng cdnditibhal 
statements, combining and separating functions (such as FIRST and BUtFIRST), and the 
use of MAKE for setting variables to values other than the keyboard Input, they also had 
miscohcQptions about the use of variables: they frequently used the same variable name 
more than once (thereby erasing all but the last value), printed a variable inside a list (such 
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that tt dioM riot git iviluiteci^ arid confused differiht variable harries. Students iri both 
grapHIci and ilat-proconlng mis-ordered commands and made Inappropriate subprogram calls 
(jneiading uniritended recuraive calls). 

4.2.4. Debugging during programming 

The frequency of test runs was calculated to rnbhitbr how rriuch the students actually 
make use of debugging in their own programming. Students tested graphics programs 
frequently, about 6 tirnes for every prograrn unit they accornpNshed. Because of the 
frequent testlrig, bugs were almost always in the most recently written commands so the 
debugging search was minimal. Frequently, the did not try to choose the correct command 
(especially the correct argurnent) on their first pass; InstLud, they used debugging as a 
means of determining the appropriate command. In contrast, list-processing groups tested 
their prograrris less than once per unit accorripiishs^d because they spent rnore tirne 
cbrnpdslng the phrasing of their PRINT statements. Because of this Infrequent testing, they 
often made the same error many times and had to correct muitlpie bugs after each test. 

4.2.5. ihdepehdehce 

Students iri both rriirij-cburses needed considerable help decbrnpbsing the prograrn goals 
Into mahLgeabie units, particularly on the iist-prbcessing tests where they had to decide 
what needed to happen in what order. The experimenter intervened to help with 
decbmpbsltlbn 2 to 3 times fbr rnbst pairs bf students. Decbrnpbsltlbn was less bf a 
problem for students on graphics tests, especially because they made few attempts to 
structure their programs: However, graphics students had particuiar difflcuity deciding how 
tb use a repeat staterhent tb draw the sun and deciding wheri tb use variables. List- 
processing students had the most difficulty with recursive stepping through a list (used on 
the SCRAMBbES test) and with deciding on the aiterriate paths iri a cbnditibnai staternent. 
Much bf the structure used in the list-processing programs was created with the 
experimenter's help. 

The students alsb heeded considerable help with twb aspects bf debugging. They heeded 
the most help correcting errors that had resulted from their misconceptions, in other words, 
in cases where, their knowiedge as weii as their program needed debugging. Students aisb 
heeded help locating bugs in cases where they Had used little prbgram structure and did 
not test the program frequently enough to know which part of the code was buggy, in 
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these cases, they may have known what was wrong, but could not find any clues for the 
bug's Ibcitlbn. The next section will describe the debugging skills students dernonstrated In 
situations where the programs were well-structured but had been written by someone else. 

The quantitative ar^d qualitative description of the students' prdgramrning abilities 
emphasizes that these students are still only novice pro^rammiirs, even at the end of 50 
hours of experience. Even though they were able to use program structure to guide their 
debugging of the experimenter's prbgrarns, they did hot Incorporate such structure Into their 
own programs. ThMs, they had more difficulty debugging their own programs than someone 
else's. 

katz (1^86) studied debugging on one's own versus another person's programs in LISP 
and found heiairiy opposite results. Re fbUhd that subjects needed twice as rnahy hints 
when debugging someone else's program as when debugging their own and that they were 
more likely to use a backwards strategy (run the prograrn to see what's wrong, then check 
each command from the last backwards) than a forwards strategy (check each cbmrhahd in 
order of execution) when debugging someone else's program^/illian their own. The 
advantage for debugging bhe's own prbgram may be less prbhbune^^^Jor the child novices 
in this study because they were not writing well*structured prS^patfisr- whereas the 
experirnehter's prbgrarns were well-structured. Korsbn and Vaishnavi (t^^ atotr presented 
evidence for the greater ease bf debugging structured programs in BASIC, if the subjects 
had been taught techniques for good program structuring, their debugging skills might have 
been as gbbd when debugging their own programs as when debugging the experlrnenter's. 

Not surprisingly, tne bugs whicli subjects failed to find In the first mrni-course were usually 
related tb miscbnceptibns that appeared in the prbgrarnmlhg tests. Oh the graphics tests. 
2/3 of the subjects failed to find a wrong sUbprbgram call (in FARM), half failed tb find the 
orientation bug between the two curves In the seagull, and 1/3 failed to find the extent bug 
In the curved tbp bf the silb. Several bther bugs were rnlssed by bnly one pair of subjects; 
usually because they ran out of time. 

Oh the ilst-prbcessing tests, the bugs subjects failed tb find were again related tb 
misconceptions they had demonstrated on the prbgrammihg tests. There were three 
instances where a wrong variable name had been used or a variable name had been used 
twice; in all three cases, about half of tPie pairs failed tb correct the bug. A llttie less than 
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half the paliv failed to find bugs dealing with conditional clauses, cbmblnlhg LAST and 
BUTbAST, printing virliblei Inside lists, and forgetting the parentheses around print 
statements. Once again, several other bugs were missed by oniy one pair; 

4.3 Editing skills 

The goal of the editing analysis was to document the improvements in students editing 
si<iiis so that they can be cornpared to the imprbvemehts In debugging time. This 
cdrnparlsbh will provide a better estimate of the actual Improvement In debugging time 
excluding the tim^ It takes to rnaneux'^r within the LOGO envirbnrnent. 

The videotapes for the editing tests were transcribed keystroke by iceystroke and the total 
tirne was recorded frbrn the lap counter on the videotape. Experimenter Inierventldn was 
also transcribed. 

4.3 i. Speed 

Editing time was calculated as the totai tirne up to the beginning of the correct test run 
divided by the number of keystrokes. The time/keystroke should decrease as the students 
become more familiar with the keyboard. 

Figures 21a and b show that students were slower at editing llst-prdcesslhg programs than 
graphics programs (F [1.128] ^ 8.26. p < .01). though students improved in both courses. 
F (1.128) =» 7.38. p < .01. The irnprovernent represents about a 15% decrease In editing 
time. Despite the imprdveitients. even the best students are only typing at a speed of 60 - 
80 keystrokes (not words) per minute. 

insert Figure 21 about here 



These increases in speed cannot account for the whole increase in debugging speed. 
The debugging time per bug was 2 or 3 minutes lower on each of the graphics tests and 
about n minutes on each of the llst-prbcesslhg tests in the second mini-course. This 
improvement Is roughly a 30% savings for graphics students who had previously had list- 
processing and a 50% savings for list-processing students who had previbusiy had graphics. 
Figure 22 shbws the debugging tirne per bug minus the editing tirne per bug fbr each test. 
There Is still a marked Imprdvement in debugging speed between the first and ^tecond mini- 
courses. 
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Insert Figure 22 about here 



4.3.2. Efficiency 

Efficiency of editing was measured In terms of the total num^ilr of keystrokes made 
relative to the inlhlmum number required. The minimum nurib^r was calculated using the 
following guidelines. Degraded forms of commands which v^^fe net taught yet are accepted 
by the LOGO '^>#rpreter were hot considered. These cor/mahds actually take fewer 
keystrokes and may be discovered by the students. For example, LOGO will Interpret ED 
just as It interprets EDIT; however, since LOGO Is not always so forgiving (especially about 
spaces and paired brackets), the instruction stressed proper form. Also, sho'^^uis, such as 
those to get to the beginning or end of a Hne directly rather than repeating tha arro^ ^<sys, 
were only considered when it was bbvlbus that using thein would be quicker, i.e*, wr^^*^. the 
bag was at the extreme end of the line. 

Students In the list-processing course were less efficient e<<'ior8 as well as being sidwef 
typists, F (1,128) - 8.04, p < .61. They averaged a ilttie mom'-t^m twice as marry 
keystrokes as necessary while the graphics group averaged bhiyr.i.6 t^^fim-the mlhlmurh 

_ ___ _ ^-j^ __i . _ ^ 

number of keystrokes. (See Figure 23:) By the second mini-course, aifc Qf< ttV9*stude had 

imprbved (F (1,128) » 39.96, p < .01), rec.chlhg the same level of editing efficiency close 

to the perfect score of 1. Thus, the students who began with list-prdcesslng improved more 
than the students who began with fraphics, F (1.128) » 4:79. p < :b5. 

insert Figure 23 about here 



Each divergence from the minimum path was categorized as one of eight types of 
Inefficiencies: overshooting (and backing up), mistyping (and correcting), deieting too much 
(and retyping), placing the cursor ihcbrrectly (and cbrrectihg). ihcludihg extra spaces (and 
deleting them where nc^resoc.ry), forgetting part or all of a change (and going back to 
cbrhplete It), confusing edltirg commands or princlpies, and taking extra steps, which uiaaliy 
means either not using a short-cui when it would save a lot of keystrokes or moving around 
in the editor after the changes had been made. Errors were categorized In this way in 
order to discover whether certain errors dropped out while others persisted or whether 
improvement of all errors was uhlfbrrh. 




Tfie studihti actually make more editing errors on the second and third tests than on the 
first test, perha^ becaase they are particalariy carefai on the initial test when they were 
jast learning how to edit. Total keystroke etflclehcy does hot rise with the hurtiber of 
inefficiencies biciuse students may realize their error sooner and so avoid large errors. For 
example, as students become bolder at asing the repeating key to rnove across the screen, 
they may actaally overshoot the target rribre frequently, but rriay hot dvershddt It by as much 
as before since they are more aware of repeating nature of the keys. Students just 
iearning to edit often do not realize they are rnaklng the key repeat so they overshoot by a 
lot before they even realize it. The students' most cdmmdh errors were overshooting, 
mistyping, deleting .too much, and taking extra steps: Ust-prbcessing students also confused 
rnany cornrnahds oh the first two tests. Students rarely added extra spaces, misplaced the 
cursdr. or forgot to make part or all of a change. As students editing efficiency improved, 
the error rates for ail types of errors dropped. 

4.3.3. Incorrect changes 

The nurnber of Incorrect edits was tallied as a measure of how careful the students are to 
check the corrections they make. (See Figure 24.) Neither group made many incorrect 
edits, which are errors In changes the students do hot catch before leaving the editor or 
edits they forget to make, in the first mini-course students only made about 1 incorrect edit 
per 6 changes; In the second rnini-cburse this dropped almbst tb zerb, F (1.128) » 7.14. p 
< .01. Even in the first rhihicburse, studeh:s are careful editors though they are hot 
particularly efficient or quick. 

insert Figure 24 about here 



4.3.4. Experimenter help 

On the first editing test, subjects In both groups needed the experimenter to help them 
(rerhihdihg them bf cbmrhahds« for example), but the arhbuht of help heeded deci t^sed Tor 
both groups in the second mini-course. F (1.128) = 35.46. p < .01. (See Figure 25.) The 
iist-prbcessihg students needed rnbre help than the graphics students in the first rnlhl-cburse 
(F (1.1281 = 5.55, p < .05). but students in bbth groups heeded little help from the 
experimenter by the second mini-course, thus, the savings was larger for the list-processing 
tests. F (1.128) « 4.01, p < OS. 
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insert Figure 25 about here 



4.3.5. Graphics versus list-processing 

the consistency of the difference between the students learning editing In a graphics 
environrnent and those learning editing in a iist-brocessing environrnent was surprising, 
especially since they were using Identical cbmmahds. Also, the groups' performance on the 
debugging and programming tests does not lead to the conciusion that the groups of 
students differ. It may be the case that editing Is rnbre difficult In a llst-prbcessing dornaln 
because of the added syntax difficulty which may cause a greater processing load. Also, 
both the commands arid the arguments are longer so there rnay be rnore chances of 
misreading or mistyping even though the rtilhlmurti number of keystroRes Is equivalent. 
Another possibility Is that iist-processing biurs the distinction between the output [lines of 
text] and the prdgrarti [lines of text]. The consistent finding ihat both groups of students 
reached the same level of editing skill lends support to the suggestion that learning editing 
In the context of llst-processlhg is difficult, it is not more difficult to edit lls^ processing 
programs than graphics programs once the editing skills have been learned (as demonstrated 
by the good perforrnance of the graphics group in the iist-processIng course), but learning 
the skills Is easier In a graphics ehvlrohmeht. 

4,4 Acquisition Summary 

In summary, the savings measures used In each of the skill acquisitlbh analyses provide a 
context in which the level of skill acquired can be assessed, the large savings on the 
debugging measures Indicates that students did begin to acquire the rnodei's goal structure. 

The importance of the relative difficulty of the tasks Is clear from the consistent pattern of 
perfa jahce across varlbus measures of the sarne skill, and across rneasures of different 
skills. The most prominent instance dt this effect is the large .-^acremerit in performance oh 
the second list-processing test: This test required the use of FIRST-BUTFIFRST recursion, a 
cbiidltlbnal stbp staterneht. and a counter, ail bf which were reiativeiy new concepts. 

The current study was not designed to evaiuaie the relative difficulty of various tests but 
rither tb show the savings fbr students taking a test after having had previous tQQQ 
experience and debugging experience. It is clear frbrh the irnprovement between the first 
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mini-cbdrae and second mlni-coaiBe In both graphics and lists that iearning did tako piaeo. 
Howivah it la not poailbla to trace the wlthlii mlhl-cburse Imprbvement. But, It Is clear that 
the debugging skiiia students have acquired are abstract enough to transfer to very similar 
program debugging tasks. Students in the second mini-coarse debugged more qaickiy and 
took fewer cycles than students taking the same tests In their first mlhl-cbUrse. These 
savings were attributed to Increasingly focused search. The next chapter will discuss how 
well these search siciiis transfered to non-computer debugging. 
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5. Debugging skill transfer 

The goal of the transfer analysis was to show that the focused search strategies learned 
froiTi the explicit debugging instruction In the LOGO er^ronrnent would transfer to slrnliar 
debuggihg situations not rnvdlvlng programming. 

A rnore detailed trace of the solution process was available frorn the transfer videotapes 
than from the debuggihg tapes because on the transfer tests the students were asked to 
read and think aloud. For each prbblern, the subject's discrepancy description, bug 
proposal, and bug Ideation cbmrhehts were recorded as well as the type of scanning 
strategy they used to locate the discrepancy Initially. Each line the subject read and each 
time the subject flipped back to Idbk at the plan and butcbme was recorded so that the 
search process could be quantified and the search strategy characterized. 

5.1 Pre*search 

The subject's pre-search strategy was one focus of the analysis. the number and 
proportlbh of correct cbmrnehts about the discrepancy, the bug, thi? location, and possible 
strategies were counted. Students who learn the importance of seeking cues to harrow their 
search before looking at the cdrnrnands In the tQQQ context rnay trarvtar this practice to 
the hew task. As with the cbrnputer debuggihg, students made few pre-search comments. 
However, the number of comments students made about the possible location of the bug in 
the directions Increased frbrn 9 bh the pre-tests to 16 on the rnld-tests to 22 on the post- 
tests (but of a possible 66). Aisb, the students heeded less help describing the discrepancy 
between the desired and actual outcomes; Instances of help decreased from 19 to 9 to 4 
for the three test-times (also but bf 66). Apparently, the students are paying rnbre attehtibh 
to the outcome informatibn prior to beginning their search. 

5.2 Qualitative search analysis 

Each subject's reading and simulating strategies were aisb categbrized separately as one 
of four qualitatively different approaches. Simulation refers to actually Interpreting the 
directlbh to determine what effect it would have: usually this process involves referring to the 
diagrams bf tables. Students were reading and thinking aloud sb their search jsrbcess was 
easily traceable. The worst strategy is to read or simulate haphazardly (a few lines here, a 
line br twb there) br tb sirnulate hbthihg (subjects rnust at least read something in order to 
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make a correetfon). This strategy Is unlikely to successfully find the bug because it may or 
rtiay not even read the buggy line let alone bother to check It against the desired butcbrne. 
A slightly better strategy, brute force , Is to read every line and slrnalate every direction 
(sbrne lines are headers hot directions). This strategy Is likely tb find the bug but 
processes many unnecessary lines. Better yet Is a setf-tefrtitrotlng fewte forc e strategy 
which reads and sirnalates everything antil the bag Is located and then disregards the rest. 
The best strategy Is selective focus on the apprbprlate subsection bf the prbgram only, 
maybe even only on the part near the bug. Each subject's reading and simulating 
strategies for the first pass through the directions was categorized for each of the three 
Items on the pre*, mid*, arid post- tests. Subjects may have several strategies or 
cornblnatlons of strategies at their disposal; however, their choice of strategies was expected 
tb shift tbwards more efficient strategies If they are able tb learn the prbgram debugging 
strategies and transfer them to the new domain. 

The students* behavior on the pretests was very much like the brute fbrce strategy of the 
debugging model. The predominant strategy was tb read ail the commands and simulate 

none, then tb gb thrbugh the dlrectlbhs again, simulating most of ..thern until the Incorrect 

_ _. _ 

direction was located. A change In focus on the later tests is apparent from the strategy 
classifications. Figure 26 shows the reading strategies for the students on each type of 
test. There were ho differences between grbups A and B sb the data have been cbllapsed 
across group. On the pre-test, half the students read all the lines on their first reading of 
the dlrectlbhs. Oh the mid-test and the post-test, only a quarter of the subjects read ail 
the lines on their first pass. A little more than ^ third bf the students read a!i the 
directions or most of the directions antii the bug was found and then disregarded the rest. 
A quarter of the students focused bnly bh the directibhs in the same subpart as the buggy 
direction. Even by the mid-test, most students had shifted to a more focused strategy for 
reading directions, (6j 23:33, p < ;001; 

insert Figure 16 about here 



The simulation strategy did not shift as quickly. Figure 27 shbws the strategy shift for 
each type of test. Two thirds of the students simulated none of the lines on their first 
reading of the directibhs bh the pre-test. Thus, their reading rnay farniiiarize them with the 
directions, but they have not checked the directibhs. It cbuld be argued that the students 
can remember the discrepancies between the plan and the outcome descriptions; this may 
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be true for the cohstrUctibh tests since they have simple diagrams, blit It is unlllcely to be 
true for the tables and is definitely not true for the maps which are too compilcated to 
remember. Some students had shifted to a more focused strategy by the mid-test, but half 
of the students were still not simulating any cbiniTiahds. By the post-test, more than two 
thirds of the students were using focused strategies, (6) « 33.14, p < .o6i The 
shifting pattern for different tests shows that students on the mid- and post-tests w<^;c' least 
likely to simulate cbmmahds bh the cbhstructrbn tests where they may be able to remember 
the drawings. This pattern Indicates that the students developed a range of strategies and 
know the cbndltibns under which each is apprbpriate. 

Insert Figure 27 abbut here 



5.3 Accuracy 

in addltlbh, the changes subjects made were scbred as either correct, Ihcbrrect, br re- 
writes. Fie-writes were, cases in which the subject did not isoiate a bug but rather added 
directions tb Uhdb the prbblerti arid achieve the desired butcbme. In the example described 
above, a re-write would be adding a direction at the bottom of the page saying something 
like, "Move thei coffee table to between the other two chairs/' The ibcatlbn of the change 
was scored as either correct, nearby, reasonable (on a line with an understandable false 
aiarm, such as on another iine describing a tabie in the example above), or incorrect. 
Trahsfer bf debugging skills frbrti computer prbgrainmihg would be reflected more in the 
improved location of the correction than in the accuracy of the correction itself since there 
was not instruction relevant to the particular types of cbrrectlcns: 

In addition to improved strategies, tile students made more correct fixes on later tests 
regardless of the order In which they took the rnlnl-courses. Figure 28 shows the Increasing 
percentage bf subjects whb itiade the cbrrect change In the cbrrect Ibcatlbn. 

Insert Figure 28 about .here 
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S.4 Solution time 

AlsOt the total time from when the subject finished reading a story to the time s/he started 
making a cHange was measured. Irnproved efficiency of search should decrease the 
solution time. Figure 29 shows that the tlrtie roqulred to suggest a bug decreases by 
almost half from the pre-test to the post-test as a resait of the shift to ihe selective search 
strategy, F (2,192) > 16.05, p < .51. Both the accuracy and the time figures show that 
the traveling directions were the rndst difficult to debug. F (2.189) - 19.66, p < .01. They 
were the tasks on which the figure provided the ieast information about the nature of the 
bug and on which the directlbhs weie the most difficult to sirnulate. The Irnprovemehts on 
these tasks are primarily a result of Increasing use of location clues. 

inseri Figure 29 about here 



5.5 Checking 

The total tirne to cornplete the solution was not rneasured because another developing 
strategy actually Increased the sdiutlbh tlrtie. As a result of computer debugging experience, 
students more frequentiy checlced the directions after making a change. This task provided 
ho opportunity to rd-rUh the directlbhs after rnakihg a change as is pbssible in cbmputer 
programming; however, the students attempted to re-slmulate the effect, of ttv^ change on 
their own to test its correctness: 

The nurriber of students who read and simulated lines after the tnttiai bug was Identified 
increase, steadily across tests for aii test types, (4) = 13.77", p < .05. (See Figure 
30.) This c^'fT'Dkihg strategy Is largely respbhsrble fbr the Increase In cbrrect fespbhses 
since checking the fix after It has been made can lead to discovering an Incorrect fix. 
Students rarely discovered incorrect fixes on the pre-test because they airnbst never 
simulated the effects of the fix they made. One thing that students have clearly learned 
f'bm debugging experience is that the fix may be wrong or may make tilings worse The 
nrbductlon system mbdel always instructs the user tb recheck the prbgram bnce a fix has 
been made. Even though retesting is not easy for debugging non-cdmputer directions, 
students demonstrated that they knew a very Irnportant goal: to check the fixes. 

Insert Figure 30 about here 
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Is acqoired In the tOSO- coniext were 
feaslhgljf focused search and Increased 
ch time and greater accuracy of fixes. 
\ alternative hypotheses suggested In 
maiuratlon because the older half of 
lemonstrate better seircli or checking 
ects were significantly faster than the 
< .05), but there was no mld-tesl or 
not be directly tested because of the 
18 absence of counterbalancing within 
e less accurate on the traveling tests, 
9r two test tj/pes were nearly equal In 
ilffefeiices between test srsslons were 
}re not. Ttie best evidence, however, 
sf^' is that the search strategies were 
r-^^-^ v; both-fts computer and non- 



6. Conclusions 

the dream of finding transfer of hlgh^level' thtiKIng sHs from computer pfogrammlhg has 
rapidly become a nightmare of mixed results because of failures to match the skills students 
learned with the skills target tasks required as well as because of poor methodology. The 
goal of this research was to demonstrate the possibility of achieving high-level transfer of 
debugging skills when the relevant skills have been appropriately specified, actually learned 
In the source domain, and directly useful iii the target domain. Thus, the approach used 
ior this research had three phases. First, a model of debugging skills was instantiated as a 
production system to specify the debugging skills students would need to learn In order to 
debug well. Since students did not learn these skills spdntanedusiy In a pilot study, a 
curriculum was designed to teach students the model's knowledge and goal structure 
explicitly. Students' debugging skills were assessed during each of two mInl'Courses and 
iheir performance in the second minkourse was compared to the performance of students 
who took that mini-course first. Debugging skills were also assessed on non-progranimlng 
tests designed specifically to require similar skills to program dehugging. Thus, the 
transferability of debugging skills from one tOSO programming dornain to a second tOSO 
programming dcmain and to lidnf fdgfammihg domains could be assmd. 



S.I Support for the thesis 

The thesis that this dissertation attempts to lest can be stated simply as fdiidws; 



Children can learn high-level thinking 
component skills are precisely specified 



skills from computer pfogramming If the 
and taught directly, 



Once the skills have been learned, they are available for transfer provided they 
are recognizable as relevant to the target task, 
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The most direct support lor the thesis Is tiiat the students who were taught debugging 
explicitly In the context of a LOGO course (either graphics or list-processing) did learn 
debugging skills and did transfer them to a second LOGO mini-course and to ndh« 
programming taste requiring debugging of directions. Their acquired focused search 
strategies for debugging LOGO programs which translered from one LOGO coni> the 
other and from the LOGO context to a non-programming context. The deb : 
learned In the llrsl minl-coDrse were easily recognizable as relevant to the : 4 
course; In fact, the comparison shows a large saving! Also, the students requested the 
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debugging charts tvih befbri dsbugglhg wai Introdueed in the seebhd niihl-cbursa so they 
appirently racognizad tha potential usefuiness of the siciiis they had used before. 
Debugging Is cbriipdsed of many sub-skiiis which may be learned and transfered to different 
degrees depending oh the particular experience and tasks. Only the gbal structure of 
debugging was directly relevant to the non-programming task used In this study (and none of 
the programming and editing skills were predicted ib be relevant). Structural clues were 
relevant tb the extent that the dlrectlbhs cbhtalried headings recbghizabiy similar to 
subprograms, the strategy shifts observed on the transfer tests Indicated increasing focus 
on the part of the directions containing the bug (use bf structural clues), decreasing 
tendency to simulate Irrelevant dlrectionSi and Increasing attempts to retest the directions 
after the fix was rnade. 

tl<e learning of transferable debugging skills In this study was achieved by adding only a 
srnall amount (1/2 tb 1 hbur) & explicit Instructlbn. Once the desired skills had been 
specified by writing the cdmputef simuiatidh model, the curriculum could be designed and 
Irnplemented easily. Only gentle rerninders to practice the debugging skills, the presence of 
the debugging posters, and the experimehter derhbhstratihg the skills were necessary on a 
continual basis after the initial explicit Instruction. It Is not necessary^ta laach a whole 
course on debugging In order fbr the students tb learn transferable skip; It 6 Important, 
howeveri for the skills arid the knowledge they require to be made explicit and to be used. 
Perkins and Martin (1986) made a similar suggestion for remedying students early dlfficaltles 
with cbmputer programmihg. They are currently testing the hypothesis that students' fragile 
knowledge can be boistefed by teaching them to use metacognitive strategies to guide their 
behavior. As for the debugging strategies, the explicit Instructlbn would be minimal, but the 
cbntihued practice wbuld be essential. 

the learning and transfer of debugging skills did not depend on the children first 
becbmihg good programmers. Since the gbbd debugging strategies bypass the need to 
interpret every command (which the pilot study show^ri chiidr*?n did poorly), debugging can 
proceed without great prbgrarnming skills (though sorne of the rnost difflcalt bugs tb cbrrect 
were also areas of difficulty In programmihg). Being able tb debug before being able tb 
program well is especially important since more bugs are generated by novices, kessier 
and Anderson (1986) agree that debugging can be learned prior to programming; they 
taught LISP students tb debug sirnple functibns befbre having experience prbgrarnming thern. 
Bassdk and Hdiyoak (1986) also suggest that "ihterdbmaih transfer heed hot necessarily 
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awiK ichlivimiht of exceptioiial expertise In the base ddmeJn''; In other words, the 
probiem-soMng procedares can transfer to a target domain even when they are not always 
executed fliwieiity In the source domain. This possibility applies directly to the transfer of 
the debugging goal structure even though students did not know all of the discrepancy-bug 
mappings for programming yet; 

In addition to these positive resuits, other results suggest ihat the converse of the claim Is 
also true: 

• there will b^ no transfer of available skills that are hot relevahi to the target 
task. 

• Without direct Instruction high-level skills are not likely to be learned a' 
therefore, cannot transfer. 

the results of the programming and editing tests show that the students did improve In 
both areas, these skills were useful in the second mini-cburse and resulted in a large 
savings. However, the transfer tasks were not designed so that these skills would be useful, 
betaiied analysis of programming and editing skills could reveat.tsoto. to which these sklK^ 
might transfer: however, that Issue was hot the focus of this^attifp:. «~ OSm research has 
shown that programming skills do transfer to relevant tasks soctr «v. angle estimation 
(Garlick. 1984) and figure comparison (Clements and Gullb. 1984). Similarly, while the goal 
structure and structural clues for debugging trahsfered to the hoh-programmlng task, the 
discrepancy-bug mappings for programming were not applicable and, therefore, did not 
transfer. In fact, only a subset of the discrepancy-bug mappings were relevant for transfer 
from graphics to list-prdcessing or vice versa. 

the LOGO skill tests also revealed that other high-level skills did hot develop as 
debugging skill did. the most difficult problem students had was wUh decorr^nosing the 
prbgrammlng problem In order to plan an appropriate program structure. ThcM. they had 
difficulty deciding which program schemas to use and how to direct the flow of cdritrdl. In 
fact, the level df prdgrarti structure was low unless the program goal required it and the 
experlrnenter offered rnany suggestions. the students also failed to acquire debugging 
strategies that vvere not included in the specific ihstructibh. such as ddcumentlhg prdgrams 
and using print statements as markers while testing prdgrams. Students alsd rareiy used 
externai inforrnation (such as syrnrnetry clues frorn other parts of the prbgrarn or bulletin 
board reference) to ease their problem sblvihg. 
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THiit nme students participated In two biher stadies: the results support the notion that 
bM\\b that have not been learned In the source domain cannot be expected to trahr er. 
Yant (198^ compared these students' planning abilities to those of a control group with no 
LOGO experience. Her task involved directing a robot to tidy a robrn under various 
constraints (a one-armed robot with lots of gas versus a three-armed robot wlth^ limited gas). 
She found that both the LOGO and the no-LOGO groups used poor planning strategies. 
The LOGO group was better able to give a iist of directions without getting lost, however, 
thase results are not surprising since the LOGO group was already described as having 
difficulty plahhSng and since giving directions 1$ clearly a skill with which LOGO students 
have vast experience. 

Dunbar and Klahr (1986) found that these sarne LOGO students were no better or worse 
than students who had hot had LOGO at discovering how to use the REPEAT key on a 
programmable toy (Big trak). The fact that LOGO students had no advantage Is not 
surprising since students do not have to discover how Individual cdrnmahds operate In a 
LOGO environment; they are told what arguments are required and what effe<^ they have. 
IHowever, the LOGO students were expected to be at a disadvantage slnc^ i use of 
REPEAT In a LOGO envirbrirneht Is a nilsleadihg ahaldgy for the Big Trak. 

6.2 F^osslbilitlBS for strengthening th@ evidenci 

Being able to docv^hieht the acquisition and transfer of specific debugging skills Is strong 
evidence in support of the thesis. However, the evidence could be stronger in the foiiowing 
ways. 

First, the results should be replicated with other students at other schools with other 
teachers to show that the effects are general. Also, the design could be improved by 
having various control groups either without LOGO experience or In a LOGO class without 
explicit debugging instruction. A study including a comparison of two groups, one with 
computer first semester and study nail second and one with the reverse schedule Is already 
planned for the coming fall. A second planned study includes LOGO classes from the same 
grades at the same school with and witPiout explicit instruction in debugging: 

The tie between the learning and transfer of debugging skills could also be made more 
direct. First, assuming that students could be taught to give better protocols, individualized 
testing could be used so that the learning path for individual students could be compared to 
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eicfi atudent's level of transfer. A strong cbrrelatlbh of Indlviduat learning end transfer 
scores would be strong support for xm thesis. In addition, the learning path could be 
traced better by counterbalancing the tests within each mini-coarse. 

Another important extension would be to strengthen the transfer effect. Some of the 
LOQQ students in this study did not shift strategies as a result of the debugging instruction 
t*^^i'haps more instructlbh, examples, and/br practice wbuld facilitate the shift for those 
^rjdents by Insuring complete and abstract learning (Smith, 1986; Bassok and Holyoak, 
V986). Several researchers have suggested that giving a hint that a previous strategy is 
relevant would Improve transfer (Gentner and Gentner, 1983; GIck and Hblycak. 1983; arid 
Holybak and Kbh, 1986). Transfer rnlghi be Improved by giving students a hint to use their 
debugging strategies. V\^hen discussing vviiether expbsure tb the LogbWrit^r mlcrbwbrld 
would lead to better writing sklMs in general, Papert (1986) commented, 

if you s^e^jransfer as an ^ctc^matlc process that needs no encouragemcr frorn 
you jthe teacher],^ it may or may not; But 1^ an^ convinced that your Imagination as 
a teachc^r will show you how to use the togoWriter programming as a transition to 
pore writing. 

Vet another possibility Is id show that the approach wbrks for dtftw. High-level skills too. 
problem decomposltlbn and structured programming have been shovtrr ta be very difficult for 
children yet are crucial fbr gbbd programming (and are helpful for good debugging). Fisher 
(1986) is developing a model of how expert Drogrammers decompose a problem Iritb a 
programming plan. Such a model vvbuld be useful for designing explicit instruction in much 
the same way as the Career md Kiahr (1906) debugging model was. Again, skill acquisltloh 
cbuid be assessed and transfer to tasks with recognizable simitarity measured. 

Finally, there is strength in predictability. The model must be developed tb the extent that 
it can predict the amount of transfer to various tasks depending on the relevance of the 
learned skills. Fbr example, we found differences between transfer tasj<s where the 
directions had subprogram-like heading and where they did not. IHbwever, these differences 
were not tested directly with controlled contexts: The goal must be to design transfer items 
with predictable differences and then tb test those predictions. 
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6*3 Applyirig the findings arid the approach 

Attempts must also bo made to apply these claims directly to educatbrsi both those 
teaching programming and those teaching other subjects. Curriculum materials based on 
the current findings could be developed for use In programming courses, and the possible 
transfer effects could be specified In terms of the educational objectives actually used In the 
schools. This dlf^^ertatlon showed that students could learn focused seareh strategies In a 
LOGO context and transfer them to a non-programmjn(^ context. Such skills might also be 
transferable to other complex search tasks typically encountered In school, such as locating 
a partlcur'::r eiitry in a dictionary or other reference book and scanlng text to find a 
particular piece of lrifbrm;^r>bh to answer a questlbh. 

For more general apr^iicablilty, th'^ anaiysls/instraction/assessment approach used here mast 
be described thorough Wtov i XY-^' educators arid researchers In other fields can apply It. 
This dissertation resirarcn r^as i^n'iwn that using a performance theory to guide Instruction 
and ass<fi?ssment can lead to realizing the drearn of 3:^dents le^rnlrrg transferable debugging 
skills. Yet, reality can only conform to the dream as researchers continue to explore the 
nature of transfer and educators begin to utilize the findings in the classr^orn. 
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Tabto 1: bl8cr»pancy-Bug Mappings in \hm GRAPES Model 
Iraitvce of down** 

Sii« **thit Bni - too iong^ Diitanci FO n or BK n 



SpTMd "ihoff «ro too cbso Anglv Lt n or Kt n 

togothor^ or or 

Oistihci FO n or BK n 



tocMoit **thls is tappesod to bo DisUnco FD n or IK fi 
In iniddio'' 

Extont "i^o too many squoroo" itorolion REPEAT n 

or 

Rocurslon stop IF :x « n 

or THEK 

Roeuniofi itttorvsi MA^f^' :x n 



Extra part '*lt drtw 5 Utio P#h pbiltidh PU difWid 

or or _ 

Progrctii caU Ex«*s cal 



Urmq part 'I^t^r*^ itistoad of Program caH SwitcHod caH 
a staik" 

Missing part '*> wantod a box thoro'* Pon position PO otnittod 

or or 

Program call CaM omtttod 



Prmt variablo prm«ad *scoro* Histaad Punctuation Qubtod variablo 

tha maLmbor** 

Not matching ' ' put tho right answor Hosting ^f^'^I ^r 

and It marftod it wrong" REAOUORb 



Orong vaiao *'lt rr^ad tho numbor Variablt nama Qrong variablo 

inftoad of tho placo" nama 



How to ERROR ME3SA6E Punctuatlbh Missing punctuation 



Uhat to ERROR MESSAGE Command Missing command 

or 

Missing prtnthoiis 

No vaHii ERROR MESSAGE Initiatizatibn HAKE "n^.rio v»»y t 

o:' 

Hd paramotfrr 

Don't know "this miss" ? 7 
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Table 2: Example Trace of the GRAPES Model with High information 
[*j start 

top-goal: test FARM 
RUN 

Run the program FARM, -->blc 
MATCH 

Did the outcome match the plan? (yes or noj? — >no 

1) test-i^ 

goal-t: evaluate FARM 
I 2) evaluate-2. 
j goai-2: describe FARM 

eONTRASf 

What type of discrepancy is there? 

[graphics or lists] ~>graphlcs 
CONTRAST 

Did you get an error message? — >no 

I I I 3) describe-1. 

J J_ 1_ goal-6: describe FARM 

CONTRAST 

What is a discrepancy between the plan and butcbme? 
[orientation, size, spreadt Idcatibh, extent^ 

extraparty wrongpart, missingpart, br 7] — >vrdhgpart 

j 1 j j 4) describe-2. 

I j I ] goal-7. propose '^^M 

The program is probably calling the wrong subprogram. 
EXAMINE 

What is he subp r ;:rar: that actually ran? 

I I I I I 5) f-opose-i3. 

I I I I Goal succe5.s£ul. 

j I I Goal successful. 

I I Goal successful. 

J__ |_ _gbal-3: represent FARM 

RECALL and EXAMINE 

Dbes the FARM program have subprograms? — >f 

I I I 6J represeht-2. 

I__ gbal-8: specify FARM 

EXAMINE 

Is the bug in a REPEAT or IF statement? ~>yes 

I I I I 7) specify-3,_ 

J L I I goal -9: specify FARM 

EXAMINE 

Wh>.h? ~>REPEAT 

I I I I I 8) speciEy-4, 

I I I I Gbal successful, 

j I j Gbal successful. 

j I Goal successful^ 

I I gbal"4: find FARM 

1119) find-8. 

I I I gdal-l-O: find farm 

The bugisfCORNjin FARM. 

((PU) (SETPOS (1305_i50J_iB)j_?PDJ_i[LT_£90)) (FD (260)) (BK (10)) (RT (90)) 
(REPEAT (4) XXX (CORN) (RT (90))_(FD (25)) (LT (90)) (B)) (PU) (FD (10)) (PD) 
(SILO) (PU) (BK (50)) (PD) (RT (90)) (BARN)) 
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I j j 1 10) find-6. 
I I I Goal succissful. 

I I Goal successful^ 

l_ 1_ goal-S; change FARM 
GENERATE 

Hdv should the fix be made? 

(change, delete, or insert] >chanKe 

I I i 11) change-1. 

I J [_ goll-11: change FARM 

GENERATE 

What should the (CORN) have been? ~> (STALK) 

I j j j 12) change-2. 

I I I I goal-i2; replace FARM 

ENTER, SKIP, DELETE, INSERT 

((PU) (SETPbS (13b) (50) (B)) (PD) (LT (90i; (FD (260)) (BK (10)) <Rt (90)) 
(REPEAT (4) (STALK) (RT (90)) (FD (25)) (LT (90)) (B)) (PU) (FD (10)) (PD) 
(SILO) (PU) (BK (50)) (PD) (RT (90)) (BARN)) v ^ ^ 

I ; I I I 13)_replace-l. 

I I I goal-13: test FARM 

RUN 

Run the program FARH. -->bk 
MATCH 

Did the cdrrectiba_ fix the problem? ~>ve^ 

I I I I I I 14)_test-2. _ ^ ^ 

i i I I I I goalrl4: evaluate FARM 

I I I I I I i 15)_eyaluate-3. 

! I I ! I I I goal-15: test FARM 

MATCH 

Did the outcome match the pi: ;1? [yes or ho]? >yes 

I I I I I I I i -Cj test^3. 

I I I I j j I I goal-16: evaluate FARM 

I I I I I I I I I 17) evaluate-1. 

I I I I I j j I Goal successful. 

I I I I I j j Goal successful. 

I I I I I I Goal successful, 

I I I I j Goal successful. 

I I I j Goal successful. 

I I I Goal successful. 

I I Goal successful. 
1 Goal successful. 
Goal successful. 

Top Goal Successful. 
No Prbductibhs Applicable. 
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Tfble 3: Example Trace of the GRAPES Model with bow Inforn^atlon 



["^Jstart 



I I ! I 

I I I I 

I I I I 

I I I J 



top-goal: test PARH 
RUN 

Run. the prdgran FARH. 
HATCH 

Did_ the butcbme match the plan? [yes or ho]? 

I 1) _test-l. 

I gdal-1: evaluate FARM 

I I 2) eviluate-2. 

i_ J_ <rgal-2: describe FARH 

CONTRi .T 

What type of discrepancy is there? 
[graphics or lists] 

eONTRASt 

Dfd you get an error message? 

I I I 3) describe-i. 

I I I g6al-6: describe FARH 

CONTRAST 

What is a discrepancy between the plan and outcome? 
[briehtatior.^ size, spread, location, extent, 
extrapart^ vrohgpart, missihgpart, or ?] 
4) describe-2. 
gbal-7: prbpbse FARH 
I 5) prbpbse-1. 
Goal successful. 
Goal successful. 
I j Goal successful. 
I I goal-3: represent FARH 
RECALL and EXAHINE 

Does the FARM program have subprograms? 
I j j 6) represent-2. 
I I j goal-8j specify FARH 
EXAHINE 

Is the bug in a REPEAT or IF statement? 

Ill 7) specify-3. 

I I I goai-9: specify FARH 

EXAHINE 

Is there any command that could be used as a marker? 
8) specify-5. 
goai-iOj specify FARH 
I I I I j I 9) specify-7. 
I I I I I Goal successful. 

I I I I Goal successful. 

I I I Goal successful. 

I I Goal successful. 

I I goal-4: find FARH 

I I I 10) find-i, 

I I I goal-ii[ find FARM 

I I I I 11) find-2^ 

I I I I goal-i2: interpret FARH 

READ and INTERPRET 



— >ok 
— >ri6 



->g]:aphics 
->nd 



— >? 



— >hb 



-->nd 



EKLC 
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jpy lifts the turtie 's pen so that no trace will Be left when it moves.) 
MATCH 

Is PU the right conmand? [yes or no] — >yes 

j j ; I I 12) lnterpret-4. 

1*11 goal-13: check FARM 

I I I I I I 13) checlc-2. 

I I j I Goal falls. 

J J ] j goal-12: interpret FARM 

READ and iNTERPRlT 

(SETPOS places the turtle at *^ 'a specified x y cbbrdihates. ) 
MATCH 

Is SETPOS the right command? [yes or rib] — >yes 

j I I I I 14) interpret-4^ 
I I I I I gbal-14: check FARM 
I I I I I J IS) check-2. 
I I I I Goal fails. 

I I goal-12: interpret FARM 



I t 
MATCH 

Is 130 the right number? [yes or no] 
j 16) interpret-7. 

gOal-15s check FARM 
I I 17) check-2. 

Goal fails^ 

goai-12: interpret FARM 



— >yes 



I I I I 

I I I I I 

I I I I 

I I I I 



i J I I 
MATCH 

is 50 the ri.<?;ht number? [yes or rib] 

III' 18) interpret-?^ 

I I I j goal-16: check FARM 

I I I I , J 19) check-2. 

I I I I Goal falls. 

I I I I gbal-12: iriterpret FARM 

HATCH 

Is (Bracket) the right punctuatibn? [yes br rib] 



— >yes 



— >:'es 



I I I I i 

I I I i I 

I I I I 

I I I I 

II I 1 



20'; iriterpret-5^ 

gbal-17: check FARM 
1 J 21) check-2. 
Gbal fills. 

gbalrl2: iriterpret FARM 



READ and INTERPRET 

CPD puts the turtle 's pen down so that a trace will be left when it moves.) 
HATCH 

Is PD the right cbmmand? [yes Or noj — >yes 

i I I I I 22) interpret-4. 

I I I I I goal- 18: check FARH 

I I I I 1 I 23) check-2. 

I I I I Goal falls. 

1 1 I 1 goal -12: interpret FARH 

READ arid INTERPRET 

(LT turns the turtle to the left a ercain n';Tiber of degrees.) 
HATCH 

Is LT the right command? [yes or noj — >yes 

I j I I I 2^) interp:et-4. 
I I I I I gbal-19: check FARH 
I II I i I 25) check-2. 
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I I I I Goal fails. 

I [ I I goal-i2: interpret FARH 

HATCH 

Is 90 the right nuaber? [yes or noj — >yes 

I I I j I 26) interpret-7. 

I I I I I goal-2d: check FARM 

I I I I 1 i 27) check-2. 

I I I I Goal 'alls. 

j ] I I gOal^i2: interpret FARM 

READ aiid INTERPRET 

(FD iiiOves the turtle forward a certain number of turtle steps.) 
MATCH 

Is FD the right command? [yes or noj — >yes 

j j j j j 28) interpret-4. 

I I I I j goai-21: check FARM 

j I j j j j 29) check-2. 

I I I I Goal fails. 

I II I goal-12: interpret FARM 

MATCH 

Is 260 the right number? [yes or hbj — >yes 

I I I I I 30) interpret-?, 

I I I I I gbal-22: check FARM 

I I I I i 1 31) check-2. 

I I j j Goal fails. 

1 i I I goal-12: interpret FARM 

READ and INTERPRET 

(BK moves the turtle backward a certain number of turtle steps.) 
MATCH 

Is BK the right command? (yes or no) — >yes 

I I I j j 32) interpret-4. 

j j j j j goal-23: check FARM 

j I I I I j 33) check-2. 

I I I I Goal fails^ 

I j I I goai-i2: interpret FARM 

MATCH 

is 10 the right number? [yes or no] — >yes 

I I I I I 34) interpret-7. 

I I I I I goal -24: check FARM 

I I I I " I 35) check-2. 

I I I I Goal fails. 

I I I I :?oal-l": interoret FARM 

READ and INTERPRET 

(RT turns the turtle to the right a certain number of degrees.) 
MATCH 

Is RT the right command? fyes cr n:j — >yes 

I I I I I 36) interprfit-4. 
I I I I ! gd?.l-25: check FARM 
I I I I i i 37) cHeck-2. 

j I I -dal fails. • 

' ; 1 i ij-a'.-12: ihterrre" FARM 

; ' _ _ ' 

.i.s Ti vi ..- number? (yes or rib] — >yes 

j I I ' : ;3) interpret-?, 

j < I ! pbal-26j "heck FARM 
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I I I 1 1 J 39) checR . •. 
| i I I Goal falls. 
^ ^ ' i ^ Sl~12» lhterp;>;;- FARM 
READ and INTERPRET 

CREPEAT executes a list b£ commands a specified number of times.) 
HATCH 

right coimahd? [yes or no] -->yes 
40) interpret-4. 
goal-27: check FARM 
I 41) checlc-2. 
Goal fails. 

gOal=i2: interpret FARM 
MATCH 

is 4 the right number? [yes or no J >y„ 

III! 42) interpret^?. 

goai-28: check FARM 
I 43) check-2. 
Goal faiis^ 
. g6al-12: interpret FARM 

Was (CORN) the appropriate subprogram call here? -->no 
I I I I I 44) interpret-!. 
I I II I goal-29: check FARM 
The bug is the {CORN) in FARM. 

(130) i50) {B)) (PD) (LT (90)) (FD (260)) (BK (10)) (RT (90)) 
(REPEAT (4) XXX (CORN) (RT (90)) (FD (25)) (LT (90)) (B)) (PU) (FD (10)5 (PD) 
(SitO) (PU) m (50)) (PD) >5T (90)) (BARN)) ^ ^ ^ ^ 

I I I I I ; 45) Che* .. 

•1 



IS REPEAT the 

I I I I I 

I I I I I 

I I I I I 

I I I I 

I I I I 



III 
I I I I 
I I I I 
i I I I 



"'il success. 



— > change 



—> (STALK) 



I I I I I 

I I j J Gt/iii successful. 
1 I I Goal succfeasfui. 
I I Goal successful^ 
I I goal-5: change FARM 
GENERATE 

How should the fix be made? 
[change, delete, or insert] 
I I I 46) change-1^ 
t I j goal-30: change FARM 
GENERATE 

What should the (CORN) have been? 
I I I I 47) change-2. 
1 J I I 8oal-3i: replace FARM 
ENTLR, SKIP, DELETE, INSERT 

.^Ifnil^fH®?^^^^^ (50) (B)) (PP) (LT (901) (FD (260)) (BK (10)) (RT (90)) 

$cffnf,i1 STALK) (RT (90)) (FD (25)) JLT (90)) (B)) (PO) FD 10)) PD 

(SILO) (PU) (BK (50)) (PD) (RT (90)) (BARN)) ^ n ( i) 

I I I I I 48) replace-1 . 

1 I I I I g6al-32: test FARM 

RUN 

Run tit 9 prv;y-ram FARM. 
MATCH 

Did the correction fix the problem? 
I I I I I I 49) test-2. 
I I I I I I goal-33: evaluate FARM 
i I I I I I I 50) evaluate-3. 



— >ok 
— >yes 
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I _l_ I I I I I gbal-34i test FARM 
HATCH 

Did the dUtcofie match the plan? [yes or ho]? 

I I I I j I I I 51) test-3. 

I 1 1 I I I I I gbal-35: evaluate FARH 

I I I I I I I I I 52^ evaluate-1. 

I I I I I I I i Goal successful. 

I I I I I I 1 Goal successful. 

I I I I I 1 Goal successful. 

I I I I I Goal successful. 

I I I I Goal successful. 

I I I Goal successful. 

I I Goal success tul. 

I Goal successful. 

Goal successful. 

No Productions Applicable. 
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Tibii 4: Description of the Subjects 





3 


8i2 


85 


95 


79 




MJ 


1 




87 


99 


68 




B( 


3 


d;5 


98 


99 


84 






3 


a;9 


74 


95 


73 




BR 


4 


9;6 


84 


91 


80 


IBM PC 


PE 


4 


9;7 


78 


87 


67 


Appl»IIe 


AG 


5 


9;9 


98 


99 


98 


Cominodor* 


AC 


4 


9ilO 


79 


87 


53 


TI 


AP 


5 


10;7 


78 


73 


79 


COfiimiwtfpf# 


DC 


5 


!0;1O 


96 


98 


98 


IBM PC 


BS 


I 


il^O 


95 


81 


96 




Aw. 


4 


9;6 


88 


91 


80 


7/ii 



MP 


3 


8;4 


77 


94 


68 




TH 


3 


8;7 


46 


52 


41 


Appi* lie 


Gt 


3 


8;7 


65 


71 


53 


IBM PC 


CH 


5 


8ilO 


97 


96 


99 


IBM PC 


BC 


4 


9i3 


89 


99 


86 


IBM PC 


CP 


4 


9; 10 


64 


72 


53 




RM 


5 


10;2 


93 


94 


86 




MH 


5 


10^6 


81 


38 


51 


Apple IR 


5H 


5 


iO-7 


66 


71 


28 


Apple Vc 


KB 


6 


11;1 


76 


89 


82 


RG 


6 


U;8 


93 


94 


73 




Awe. 


4.5 


9;9 


77 


79 


66 


6/11 



' At the beginning eirthiesursi. 

2 Sim tht sgbj«te «pen severe! qrmm. the scores ire f»t comjireble eeroB teeS. ^he^Srd^ 
grtJef^twfctiieStiiiftnitehievemsht Twt^Primery 3, rorm E,Comp1ete Betterj. Tjis 4th«ntf 
S*h grskn tootc tiB Steiifert fclWevejnent T«^, Internwdiete 1 , ForinJ,j^mpiete Bsttsrg. The 
6th gredire mi the Mstropolitan Achievement Test, Intermediiti Form ICS, Qimpleti Bittery. 
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tabto 8: Svquane* of Sraphia tmons 

Each gra^lhics leisbh was approximately one hour lOhg. 



ei^cs (first ninU Gf^phlcs (second Mini) 



1 


itefnonstrBtlbn 






2 




1 


basic commands 


3 


int8r)BCt1V8 project 






4 


name projects 


2 


name proJecte/SET 


5 


REPEAT 


3 


REPEAT 


6 


^ Tvet 


4 


curves 


7 


FRSSRAn TEST 1 


5 


PROSRAn TEST 1 


8 


su&pregrams/farfn projects 


6 


subprogrBms/ fartn projects 


9 


OEfliie TEST j 


7 


DEBUS TEST f 




orlglnaf projects 


8 


original projects 


It 


El^lT TEST 1 


9 


EDIT TEST 1 


12 


original projectf 






13 


debugging 


16 


debugging 


14 


vahafili shape programs 


11 


variable shape programs 


15 


more varlaPles 


12 


more variables 


16 


PRdS^AII TEST 2 


13 


PROSRAH TEST 2 


17 


seashore projects 


14 


seashore projects 


18 


DEBUS TEST 2 


15 


DEBUS TEST 2 


19 


original projects 


16 


recursion 


26 


EDIT TEST 2 


17 


EDIT TEST 2 


21 


recursion 






22 


more recurslen 


18 


more recursion 


23 


PRdSRAH TEST 3 


19 


PROSRAH TEST 3 


24 


garden projects 


20 


garden projects 


25 


DEBU6 TEST 3 


21 


DEBUS TEST 3 


26 


original projects 


22 


original projects 


27 


EDIT TEST 3 


23 


EDIT TEST 3 
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XtSU 1: Svqtwnc* of U«t-proeming Lonera 

Each list-processing lesson was approximately one hour long. 



Lists (first Mint} Lists (second nini) 



1 


defnofistfBtl dfi 






2 


words sfd lists 






3 


pRirrr, make, ofid if 


1 


ralNt.MAKE.OfldiF 


4 


fntarvlsw projects 


2 


interview projects 


5 


pfiftting 


3 


printing 


6 


finish interview's 


4 


original projects 


7 


PRD6Rftn TEST 1 


5 


PROfiRftn TEST f 


8 


sufiprsgroms 


6 


subprogrBfns 


9 


0EBII8 TEST 1 


7 


DEBUe TEST i 


16 


modlf B pi^ects 


8 


modlib projects 


1 1 


EDIT TEST 1 


9 


EDIT TEST 1 


12 


quiz projects 






13 


debugging 


10 


debugging 


14 


veriebies/recursion 


11 


variables 


is 


vahsbles/recursien 


12 


vailablss/recyreidn 


16 


PRdSRAtl TEST 2 


13 


PROBRAIf TEST 2 


17 


unscromblerjirojects 


14 


unscromblsr projRts 


18 


DEBUG TEST 2 


15 


PEBUilTESr2 


19 


originol projects 


16 


RANOen (nurnber guessing) 


20 


EDIT TEST 2 


17 


EDIT TEST 2 


21 


RANDOn (number guessing) 






22 


iTEH/egyrrr (siiiy stones) 


18 


if En/EQUrn'(s11Uj stories) 


23 


PROSRAn TEST 3 


19 


PROERAn TEST 3 


24 


poetry^pro|ects 


20 


poetry projects 


25 


DEDU6 TEST 3 


21 


OEBUe TEST 3 


26 


originoi projects 


22 


dhginol projects 


27 


EDIT TEST 3 


23 


EDIT TEST 3 
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fibis 7: Buggy 6lr«ctloni for Arranging Furnlturi 



Nerv am Uw iir«€tlMi Mrs^ Fisher gave le the movers^ 

To arrange the cHmni reom, 

Cwiter the chira cabinet on the west waH. 

Place the sliver caiihet in the south-east comer. 

Put the tc^ie in the center of the n^m. 

Afrmg^ the 6 c^rs around the table evenly. 
To arnsige the living room, 

PiKe t^ c^ihets a^ihst the west wa^ 

Pjace &ie chair in front of each er^ of the cabinets. 

Pjace^tfe squw* tablejn the 

Put the sofa on the north wall, next to the square t^ie. 

Place another chair on the south wall, across from the sofa. 

Put the coffee table between the two chairs. 

Put the rocker on the east wail, next to the sqinre table. 
To arrange the kitchen. 

Put the refrigerator in the north-west corner. 

Put the dishwasher to the right of the refrigerator. 

Put the sink to the right of the dishwasher. 

Put the stove to the right of the sink. 

Place the counter next to the stove and along the east wail. 

Put the oven along the east wall, next to the counter 

Place the table In the south-west corner of the room. 

Arrange the 4 chairs around the table evenly. 

OTffi^ one thing to fix Mrs. Fisher's directions. 
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ngura 1: Samplo Sraphlcs Programo 

the left-hand pa^^^ list the^rev[ously program, and the right-harid panels illustrate 

the graphic effecta of the commands Jjsted. AH drawings start with the turtle jsbsitlbhed at 
the bottom left of the drawing, oriented to the north, a) tOSO programs to draw a flower 
of variable size comprised of a line, two leaves, and ten diamonds, b) Recursive bOGO 
program to draw a series of flowers decreasing in size. 
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Figure 2: Simpio Llst-procesiing Programs 



Tha top half of eaeh panal lists the previously stored program, arid the bottom half 
jllustrates the interactive effects bf_the comrhands listed, The program user typed the Inputs 
beside the rectatigular cursor. The Iriierrhedlaie llries show the program's response: a) 
LOGO program to translate brie word jrilb jjlglatiri, b) Recu^^^ to translate 

a sentence into plglatln, c) LOGO program to jriiegrate the other two Into a user friendly 
garne. 



r — 




T0Pi6EV:lll 




QUrPUT lUIQRl lUIFIRST :IU FIRST :UI "RVI 




END 






VRINTPIG6V 'PIS 




IGPRV 






) 



TORLLPlGGVa 
IF EMPTVP^:L lOUTPttT IH 
OUTPUT SENTENCE PIGGV FIRST : 
ENB 

PRINT RttPlGGV (PORKV pjSI 
ORKVPflV itPRV 



t RttPlGGV BifTFlRST :t 



TOPlGtflTIN 

PRINT (PtEflSE TVPE fl SENTENCE THAT ¥011 UlflNT TRRNSLRTEO INTO PIGLRTINI 

PRINT RttPlGGV REROLi ST 

PRiNT (iUOIILO VgU LIKE TO CONTINUE?! 

MRKE 'V REROUfpRO 

IF EQURLP :V "NO (PRINT (THRNKS FOR PLRVING. HRUE H NICE QflV!I] [PiGfcflTINl 
END 



IPlGLRTiN 

PtERSE TVPE R SENTENCE THRt ¥011 UIRNT TRRNSLRTEO INTO PIGLRTIN 

■THIS tlTn.E PIGGY ItlENT TO MRRKET 

HISTflV ItttEtRY IG6VPRV ENTUIRV OTRV HHKETHHV 

UfOUtO ¥0U LIKE TO CONTINUE? 

■¥ES 

PLEflSC TVPE R SENTENCE THRT VOU UIRNT TRRNSLRTEO INTO PlOtSTlN 

■THJS LITTLE PIGGV UIENT HOME 

HISTRV ITTLELRV IG6VPRV ENTUIflV QMEHRV 

UIOULO VOU LIKE TO CONTINUE? 

■NO 

THRNKS FOR PtflVING. HRUE R NICE OflV! 
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nflur* 3: The Qoil Strueturv of ini GRAPES Mbdii 
Soil tree for the SHAPES debugging model. (Highlighted goais explained in texr.j 




SCRIBEl S IBEMESENII 




IPROPj^H^ [SPECIFYl S tNTEBgBETl 



eHECK 




101 



Trantftr of Dcbagging Sidll 



Flgtir* 4: a Sampii Debugging Problem 

Exampio of a desired output (U, the actua[ output (b). and the bugsy prddram 
produced the flawed output (c). The bog Is the call to the subprcgram CORN in FARM 
should be a call to STALK (which In turn calls CORN). 



a; 




TQ FfiRH 

PU SETPdS t^20 ^50] PD 
LT 90 FD no BK 10 RT 90 
REPEAT 4 [CDRH RT 90 FD 25 

END 



bT 90] 



to STfitK 

FD J5 LT 90 CORN RT 90 FD 30 EDRR 
FD 50 LT 90 CORN RT 90 FD 20 BK 95 
END 



TO CORN 

REPEAT 2 iREPEftT 9 
END 



£FD 3 RT 90] RT 90] 
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Rgurt S: High information @oai trM 

Goa[ tree generat^^ high-informatlbh debugging of the FARM prbgram^ Nambers 

correspond to^ the order \n which the sub-gbais were generated Pareniheilcai elements 
indicate accumulation of information In wbrkihjj membry. 



0- TEST 
fsrni 

^(fttltCft ffOl 

f. EVALUATE 
farm 




2. DESCRIBE 3. REPRESENT 4. FIND 

fvm farni farm 

I ^pisc. graplilcs) hsobprograms?) 

^(Syntax no) ^ 

6. DESCRIBE 8. SPECIFY 1 0. FIND 

farm farm f wm 



larr 



5. CHANGE 
farm 

1 1 . CHANGE 
farm 



I (DISC, wroiif part) ^°'^> I ... . 

^ Xstructurv) J^(f Ix Is STALK) 



f 

7. PROPOSE 9. SPECIFY 

farm farm 
(bug could be corn} 



12. REPLACE 
farm 



13. TEST 
farm 

M. EVALUATE 
farm 



15. TEST 
farm 

^(mitch yes) 
ie. EVALUATE . 
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Plgura 6: Comparison of High and tow information Goai tram 

Comparison of schematic goal trees for high-lhfbrmatloh (a) and low-information (b) traces. 
Note that (a) Is the same goal tree shown In Figure 5. with much of the detail suppressed. 




• 

i 



ii 

t 

19 

I 

4 

IS 



b; 



i 




ii 

12 



10 IS 14 IS tft 17 !• 19 30... 27 MM » 

««««««J|J|...JIS 



i 

i 

31 
53 

t 

54 
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Flgura 7: A Pri.teit/Poit-i«it Trinifir SMign. 



a) ThB typical pre-test/pbst-tesi transfer design Involves testing a group of sutjjects before 
arid after ha|f bj tHem^ecelve s^^^^ b) Greater Iriiprdvemeht of the treatrnent 

group than the no treatrnent group from ihe pre-iesi to the pbst-tesi is evidence of transfer 
(assurn[ng Jhat Jhere are no confounds In the experiment), c) In the dissertation study, 
subjects were tested at tfiree times: before, during, arid after LOGO experierice includlrig 
debugging Insirucilon. 1, 2, arid 3 starid for three types of tests giveri at each tlrne: a. b. 
and c represent three differerit versloris vvhlch were used so that the tests coald be 
counterbalariced across test time. 
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ngura a: PMtms of Rosuits SuoQWtod by Mtirhitlvi Hypothwii. 

a) Oho altorhatiyo hypothesis la that transfer effects are purely the result dl maturatldh 
during jhe time between the tests. If this Is the case, then the younger subjects should 
score lower oh the pre-tests than the older studehis ahd lower on tlie post-tests than the 
birier subjects e^r were (J^ecause Ih their ages Is greater than the te^^^^ 

Interval), bj A second aiternajlve hypothesis Is that transfer effects are the result of practice 
on the tests. In this case, [rnprovements should be constant across all nine tests (three at 
each test time) rather than showing increases primarily between test times. 
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Plgiirt A Savings Transfor Dotlgn. 



A jyptefiL savings df sign jnv^^^ two groups of subjects doing jwb tasks jn^ 
orders, b) Transifef if indicated by better pjerformance dri task A by the group doing task B 
first and by better performance on task B by the group doing task A f|rsi; cj in the 
dissertation study, the two groups of sub[ects took two bOQO mini-courses (graphics and list- 
processing) Iri different ord^ |n ewh coarse^ they took three series of tests, each of 

which had three items (programming, debugging, and editing). 
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Fljgiiiv 10: A Comblnid Pri-tmt/Post-tMt and Savings Dttlgn. 

This figure Is a combination of Figures 7c and 9c; It shows tha complete design. 
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Rgura 11: The Model-Based Debugging instruction 

Debugging lnstruct[on w^^^ derived d|rectly from the model (with changes In wording f5r the 
benefit of young student^^^^ process taught explfcltiy in both 

tOQO mini-courses is represented here in terms of the goal structure of the GRAPES 
model. 
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ngurv 12: Plinnad and Bttgay OutcbmM for Arranging FumHtiro 

Exampio of the discrepancy information provided on a transfer test. The bog is the coffee 
tatjie in the jjying room. Using this clue would help narrow the search for the buggy 
direction (see Table 6). 
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Pigtira 13: A Sample babaooing Transcript 



Exampla trahscrtpt^ frorn a debooging test: AH transcriptions were done In terms of the 
model's goal strticttire. The model's goals are shown in parentheses below each example 
statement; 
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Flguri 14: Debugging Sucetii 

Group A took graphics then list-processing; Group B took the rnihi-courses in. the reverse 
order, a) Compirlion of the two groups oh the graphics tests, Comparison of the two 
groups on the llst-pfocesslhg tests, c) Overall resuit: second mlhl-cburse groups found more 
bugs th&n first mlhi-cburse groups in graphics and list-prdcesslng. The maxirnum number of 
bugs was six. 
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ngura IS: Debugging Speed 

Group A took graphic then llst-prbcesslhg; Group B took the mihi-cdurses in the reverse 
order, a) Gompartsbn oM two groups on th^ graphics jests^ b) CompaHsoh of the two 
groups on the Hst-proceMlng tests, c) Overajj^ resuiJ:^ second mJnUcounse groups found bugs 
more quickly than first mini-course groups in graphics and iist-processlng. 
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Figara 16: Oibugging Efficiihcy 

GroQp A took graphlM then list-processing; Group B took the mihi-cburses Ih^'the reverse 
order a) 6ompirSon^ o^^ two groups on the graph[cs tes^^^ of the two 

groups on the iiM-processIng tests, c) Overa[i result: second mlnl-cou^^ 
cycles (in terms of the model) to fix bugs than flrst rnlnj-course groups in graphics and list- 
processing. All students took fewer cycles on list*processing tests than on graphics tests. 
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Flgura 17: A^racy of Search Gommenta 

Group A tooM graphics then list-processing; Group B took the rtiihi-CQurses in. the reverse 
order. a) Commehti describing the discrepancy, b) Comments proposing the bug. c) 
Gbmments specifying the location. Accuracy of search comments was high; however, very 
few search cbmmeht'^ were made. There was no Improvement In the accuracy of any type 
of search comments. 
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RgUft 18: Arhbuht of Pri^irch Commeffts 

Group A took graphic then list-prdcessihg: QrbUp B took the mlhl-coUrses Irt 'the reverisie 
order. a) Commems describing thi discrepancy, bj Cbmrrierits proposing the bug. c) 
Corhmenti specifying the location. The percentage of corrirnents made prior to Iriltlailrig 
search increased, especially for cbrnmehts proposing the bug. 
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FTgui^ 19: Amount of Program @oai AchlMl 

Group A took graphics then list-processing; Group B took the rnlni-cburses in .the reverse 
order, a) Cpmpirlsbh of the two groups on the graphics tests, b) Cbrnparlsbh of the two 
groups on the llst-prbcessing tests, c) Overall result: there was small iinprbvemeht In the 
percentage of program units completed in the second rnini-cburse. Students cbrnpleted 
more of the program units on the Iist*pr6cessing tests. 
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Ftgara 20: Aftiount of Progrim Struoturi 

Group A took grarHics then llst-processlhg; Group B took the mlhl-cburses In - the reverse 
order, a) Corhparlsoh of the two jarbups bri^ the graphics tests, b)^ ebm 
grbups bh the llst-prbcessing tests, c) Overall result: second mlnl-cbarse groups added no 
mbre^tjiicture tb tjieir programs^Jhan first m|ni-cbaree grbops In graphics and list-processing, 
StQdentd in list-processing did add more structure than students in graphics, however. 
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Pigura 21: Editing SpMd 

Group A took graphics thon list-processihg; Group B took the mihi-cbUrses In -the reverse 
order, a) Comparison of the two groups on the graphics tests, b) Cornparlsbri of the two 
groups on the llst-prpcessihg tests, c) Overall result: secbrid mirii-cburse grbups were faster 
at editing than first rtilht-cburse grbups in graphics and list-processing; Students were faster 
at editing graphics prograrns than llst-prbcessing programs. 
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Fjgara 22: Otbugglng Speed Minos Editing Speed 



ERIC 



Group A took graphics than llst-prbcess[ng; Srbup B jbbk the minl-cburses l^yihe reverse 
order^ The connected p^^ the editing speed. 

For refaranco,^ the^r^cor^nected points show the debagging speed without adjustment (as in 
Figure IS), a) eomparison of the two groups on the graphics tests, b) Comparison of the 
two groups on the list-processing tests, c) Overall result: Subtracting the editing time from 
the debugging time per bug does not diminish the transfer effect. 
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FIguri 23: Editing Efflclincy 

@roup A took^ graphics th^^ Hst-processlng; Group B took the mini-courses In the reverse 
order a) Coniparifon of^ two groups on the graphics tests, b) Coitiparlsdn of the two 
groups on the jl«-prweMjng test^ c) bverall result: second rhinl-cdurse groups were more 
efficient at editing than first mini-course groups in graphics and list-processing. 
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Flgura 2«: Incorrect Er^^^ 

Group A took graphics then llst-processlhg^ Group B took the mjnl^ourses In the reverse 
order, a) Cpmparlsbn of the two groups on the graphjcs tests, b) Comparison of the two 
groups ots the iist-processihg tests^ c) Overall result: second mini-course groups made fewer 
incorrect edits than first mihi-course groups In graphics and Hst-processlng. 
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FtoMT* 21: Amodnt of Help NMd«d for EdHlhg 

GroQp A took^ graphics then list-processing;^ Sroap B took the mini-coorses [tr.the reverae 
order, a) eomparifon of the two groups 0^^^ the graphics iesis, b) Comparison of the iwo 
groups on th^o j[st>proce&sing tests, c) bverali resuit: second mini-course groups required less 
help than first mini-course groups In graphics and list-processing. 
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nsurs 2t: Riiding Stratvgin on Trintfor totts 



the number of soSli^ti who used ea^^ of the four search strategies foj- reading^ the buggy 
direction!, tj On arrtngino direcjion tests, b) On distributing direction tests, c) On trav sling 
direction tests. Better strategies are toward the right. Subjects Shifted toward better 
strategies on the mid- and post-tests; 
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Ffgura 27: Simulation Strit«o>M on Trantfor "astt 

The humtjir of iubjicti who Used each of the four search strategies for simuiating the 

buggy dirietiora (ehieking them against the out^^^^^ a) Oh arranging dlrect^^^ On 

distributing djiwitldh t^ traveMng direction tests. Better strategies are toward the 

right. Subjects shifted toward better strategies on the mid- and post-tests: 
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Flflyra 21: Succtss for Dobugolng DIrvetlont 

Oh aii thrM types of transfer test, more subjects succeeded In debugging the directions 
the rtild- and post-tests than on the pre-test/ 
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ngura 29: UrM to Dibug Siroctlora 



Oil all thrw typw of transfer test, subjects took less time to locate the bug.* oh the mld- 
and p09t«tests than on the pre-test. 
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ngwi 99i Uii of th« ChioiUng Stritigy 

For iii thrw typw of jranster tost, moro subjects checked their fixes (rather Uian quIHIno 
Immidlitoly iftor making a change) on the mid- and post-tests than on the pre-fsst. 
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